EarlyMorning - Progress & Discussion

One suite to code them all. An complete IDE and assembler for all your z80 projects!

Moderators: benryves, kv83

Post Reply
User avatar
tr1p1ea
Maxcoderz Staff
Posts: 4135
Joined: Thu 16 Dec, 2004 10:06 pm
Location: I cant seem to get out of this cryogenic chamber!
Contact:

Post by tr1p1ea »

I guess since the escape char is prefixed to the data and the decompression routine expects this, there is no need to define it elsewhere. Sounds pretty good to me.

I also dont see why you would use the most common byte?
"My world is Black & White. But if I blink fast enough, I see it in Grayscale."
Image
Image
King Harold
Calc King
Posts: 1513
Joined: Sat 05 Aug, 2006 7:22 am

Post by King Harold »

Shouldn't you use the least common byte? since it will require less double escape chars?
User avatar
Timendus
Calc King
Posts: 1729
Joined: Sun 23 Jan, 2005 12:37 am
Location: Netherlands
Contact:

Post by Timendus »

Heh, sorry for all the confusion :)

I'm talking about the specific case in which I have two buffers of full screen images for a greyscale image. Those buffers have a tendency to be mostly white or mostly black. So because there's long stretches of black of white, the runlength encoding is useful for those areas and not for the other bytes (because they usually never repeat). Therefor, I use black or white, whichever is used more in the image layer, as the run indicator or escape char and as the data for the run at the same time. I prepend this byte at the beginning of my data so the decompressor knows which byte is the indicator:

Code: Select all

$00,$00,$00,$00,$AA,$BB,$CC,$00,$00,$00,$00
(one row of some white image with black figures in the center)
-->
$00,$00,$04,$AA,$BB,$CC,$00,$04
<indicator definition>,<indicator>,<runs>,$AA,$BB,$CC,<indicator>,<runs>
http://clap.timendus.com/ - The Calculator Link Alternative Protocol
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
User avatar
kv83
Maxcoderz Staff
Posts: 2735
Joined: Wed 15 Dec, 2004 7:26 pm
Location: The Hague, Netherlands
Contact:

Post by kv83 »

I finished the RLE compression :D.

This map:
Image

is in normal data:

Code: Select all

;1. Level 1
0,0,0,0,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
3,3,3,3,3,3,3,3,3,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
4,4,4,4,4,4,4,4,4,1,2,0,0,0,0,0,0,0,0,0,0,0,0,0,
3,3,3,3,3,3,3,3,3,3,3,2,0,0,0,0,0,0,8,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,1,4,1,2,0,0,0,0,0,7,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,1,4,4,1,2,0,0,0,0,6,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,1,2,4,4,1,3,2,0,0,5,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,1,2,4,4,4,1,3,3,3,2,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,1,4,4,4,4,4,4,4,2,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,1,3,3,3,3,3,3,3,2,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
3,3,3,3,3,3,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,3,
4,4,4,4,4,4,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,4,
4,4,4,4,4,4,2,0,0,1,3,3,3,3,3,3,3,3,3,3,3,3,2,4,
4,4,4,4,4,4,2,0,0,1,4,4,4,4,4,4,4,4,4,4,4,4,4,4
and in RLE it is converted to:

Code: Select all

.db 255 ; - RLE Escape Character
.db 255,4,0,8,255,23,0,6,255,19,0,255,9,3,2,255,14,0,255,
    9,4,1,2,255,13,0,255,11,3,2,255,6,0,8,255,14,0,1,4,1,2,255,
    5,0,7,255,14,0,1,4,4,1,2,255,4,0,6,255,14,0,1,2,4,4,1,3,2,0,
    0,5,255,15,0,1,2,255,3,4,1,255,3,3,2,255,15,0,1,255,7,4,2,255,
    15,0,1,255,7,3,2,255,28,0,255,6,3,2,255,15,0,1,3,255,6,4,2,255,
    15,0,1,255,7,4,2,0,0,1,255,12,3,2,255,7,4,2,0,0,1,255,14,4,0
Note: The RLE Escape Character is always 255, except when 255 is used as value in the map. When that is the situation, the RLE Escape Character is the least common value in the map array. (Just as Jim said.) As you can see the format of the RLE-compressed values are:

Code: Select all

Compressed: <escape character>, <run length>, <value>
Values are only compressed if they occur more than two times in a row.

Compression Rates:
No compression: 384 bytes
RLE compression: 141 bytes (Compression Rate: 63.28 %)
ByObject compression: 153 bytes (Compression Rate: 60.15 %)

Of course the compression rates hugely depend on the mapData and are different from map to map. I guess the ByObject compression works best with empty maps which use not to much different tiles. I'll include a "compression tester" btw. That way you can see what compression is best used for your maps and/or sprites. (Sprites can't be compressed till now) :)
Image
CoBB
MCF Legend
Posts: 1601
Joined: Mon 20 Dec, 2004 8:45 am
Location: Budapest, Absurdistan
Contact:

Post by CoBB »

Timendus wrote:I'm talking about the specific case in which I have two buffers of full screen images for a greyscale image. Those buffers have a tendency to be mostly white or mostly black. So because there's long stretches of black of white, the runlength encoding is useful for those areas and not for the other bytes (because they usually never repeat). Therefor, I use black or white, whichever is used more in the image layer, as the run indicator or escape char and as the data for the run at the same time.
This is how I compress the surface angle information in Acelgoyobis tables, which consists of mostly zeroes when expanded. :)
User avatar
tr1p1ea
Maxcoderz Staff
Posts: 4135
Joined: Thu 16 Dec, 2004 10:06 pm
Location: I cant seem to get out of this cryogenic chamber!
Contact:

Post by tr1p1ea »

Coming along very well there kv83. RLE should be just fine for platform games, especially if there is a lot of white space and repeating tiles. I like the example map you used hehe :twisted:.
"My world is Black & White. But if I blink fast enough, I see it in Grayscale."
Image
Image
User avatar
kv83
Maxcoderz Staff
Posts: 2735
Joined: Wed 15 Dec, 2004 7:26 pm
Location: The Hague, Netherlands
Contact:

Post by kv83 »

I finished the compression info screen. If you right-click a mapCollection, you can click on "Compression Info" and it will pop-up a new screen and show the information.

Image

(The above screenshot is of the Original Levels of Slippy. A good example that RLE is not always the better method compared to ByObject! :) )
Image
User avatar
Jim e
Calc King
Posts: 2457
Joined: Sun 26 Dec, 2004 5:27 am
Location: SXIOPO = Infinite lives for both players
Contact:

Post by Jim e »

Well sure tiny mostly white maps with only 4 or 5 tiles probably wouldn't do so bad with byobject, but if you have lots of tiles or a map bigger than 16x16 than its utterly useless.

Hmm, maybe you should make use of iondecompress, since that is a library supported by the most popular shells. All it does is reduce the number of bits to store info. When there are very few tiles that can shrink data size immensely.



Can we get a tool bar to do the more common things?
Save, build, new sprite, new map, move up, move down and what not?

EDit: BTW nice window!
Image
User avatar
Saibot84
New Member
Posts: 38
Joined: Fri 17 Feb, 2006 9:14 pm
Location: Jersey City, NJ

Post by Saibot84 »

Image AFAIK, I was under the impression that $91 was being used as a marker in current calc RLE software, but I could be wrong... actually, I was wondering if it might not be better to add a small header to the compressed output, where the marker ("control char") is defined (for the decompressor. The marker is defined by the compressor after having scanned the data to see if there is any byte that NEVER appears in the data, and using this byte as a marker.
____________________
Reaktix Software (formally XDG Kat-Productions)
-Michael Image Image
MyBlog ChatWithMe
User avatar
tr1p1ea
Maxcoderz Staff
Posts: 4135
Joined: Thu 16 Dec, 2004 10:06 pm
Location: I cant seem to get out of this cryogenic chamber!
Contact:

Post by tr1p1ea »

Ummm, Sailbot84, did you read anything on this page at all? Your idea is exactly what is implemented.
"My world is Black & White. But if I blink fast enough, I see it in Grayscale."
Image
Image
User avatar
Timendus
Calc King
Posts: 1729
Joined: Sun 23 Jan, 2005 12:37 am
Location: Netherlands
Contact:

Post by Timendus »

Looking good, kv!
By the way, for the tileset and text data in The Core of Light we used Huffman and LZW compression 8) ;)

..or was it LZ77 and Adaptive Huffman..? It's been too long :?
http://clap.timendus.com/ - The Calculator Link Alternative Protocol
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
King Harold
Calc King
Posts: 1513
Joined: Sat 05 Aug, 2006 7:22 am

Post by King Harold »

although you can't really try to use a value that is never used can you? what if there is no such value? does it take care of that? in other words: does it use the first least used value? (so if a value is never used, use it and stop trying others)
User avatar
tr1p1ea
Maxcoderz Staff
Posts: 4135
Joined: Thu 16 Dec, 2004 10:06 pm
Location: I cant seem to get out of this cryogenic chamber!
Contact:

Post by tr1p1ea »

Timendus, you used the huffman compression written by Kerey Roper? (He was part of COL right?) I used the same stuff in Desolate, i still have all the stuff in the project folder i think.

Should come in handy :).
"My world is Black & White. But if I blink fast enough, I see it in Grayscale."
Image
Image
User avatar
kv83
Maxcoderz Staff
Posts: 2735
Joined: Wed 15 Dec, 2004 7:26 pm
Location: The Hague, Netherlands
Contact:

Post by kv83 »

King Harold wrote:although you can't really try to use a value that is never used can you? what if there is no such value? does it take care of that? in other words: does it use the first least used value? (so if a value is never used, use it and stop trying others)
It first tries to use 255, when that is used in the array, then it will use the least common value in the array (that could be a value which occures zero times aswell)
Image
User avatar
Timendus
Calc King
Posts: 1729
Joined: Sun 23 Jan, 2005 12:37 am
Location: Netherlands
Contact:

Post by Timendus »

tr1p1ea wrote:Timendus, you used the huffman compression written by Kerey Roper? (He was part of COL right?)
Nope, I had another genius to command :mrgreen:
http://clap.timendus.com/ - The Calculator Link Alternative Protocol
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
Post Reply