[CSDb] - User Forums - Release id #232976 : Next Level

Welcome to our latest new user Chriszx/UNITRAX ! (Registered 2024-05-26)

You are not logged in - nap

CSDb User Forums

Forums > CSDb Entries > Release id #232976 : Next Level

2023-06-20 19:57

Digger

Registered: Mar 2005
Posts: 422

Release id #232976 : Next Level

Well, well. I think that such demo deserves a thread here.

I already spoke with BitBreaker about some things, and what puzzles me is the process behind capturing 100 games by Pex. This part has absolutely crushed me during the compo – great idea and execution there!

However, I must say some game screens felt pretty static (I guess due to amount of VIC/SID data needed to be captured and played back) and some classic (e.g. Turrican) games were missing.

I was wondering if it crossed your mind to compromise on the amount of games but keep 'em on screen for twice as long.

And of course if @Mahoney would share some secrets about the capturing process it would be awesome. Thx.

... 21 posts hidden. Click here to view all posts....

2023-06-22 00:14

wacek

Registered: Nov 2007
Posts: 505

Quote: Great! Do you think you could find the link Wacek?

What you quoted was exactly what was posted on fb :)

2023-06-22 05:58

Raistlin

Registered: Mar 2007
Posts: 575

Quote: Great! Do you think you could find the link Wacek?

https://www.facebook.com/groups/2503729026388481/permalink/6497..

Pex's answer:-
"I know how it was done. I rewrote the Vice emulator to dump not only the whole state of the emulator at a certain point in time as a snapshot, but then also every single read and write to memory with a timestamp. Recording a 5-second snippet of a game resulted in a 4MB file on disk. Then, I wrote a python script to extract, as Robert correctly stated, the colours on the screen, the chars on the screen, the charset, the sprites. Every game is 2 seconds, which means 100 frames of updates needs to be tracked, so I gather a list of every single memory location that needs to be changed to update the graphics, by fast forwarding the state of the C64 SID and VIC-II chips some 19000 clock cycles to get to the next frame. This leads to a 100 to 200 lists of memory locations with 100 values that needs to be written to replay the graphics and sound. I need to compress these, but more importantly, I need to decompress these lists in real-time with as few CPU cycles as possible, since I only have some 19000 clock cycles until the next frame needs to be shown. And then I added a sprite multiplexer to show the texts on top of that. It's easy when you look at one single part of this at a time, but really messy when you combine all needed parts - I need to load the data for the next game while running the current one, for instance."

So, yeah, as Wacek mentions, it's word for word what you posted above. Nothing sinister implied, if Pex sent you the above then he's either copy/pasted from his FB post, or he's copy/pasted -to- his FB post.

2023-06-23 04:26

Pex Mahoney Tufvesson

Registered: Sep 2003
Posts: 50

The largest games on disk are Bombjack (3167 bytes), Giana Sisters and Revenge of the mutant camels, all larger than 3kB.
The most difficult game to get working was Thing on a spring, which contains a lot of animated graphics and I need to update 138 memory addresses continuously. So, during Lazy Jones, I need to load 2.5kB from disk and unpack to 12kB. I was forced to place Panic 64, one of the smallest games, directly after Thing on a spring. (636 bytes to load, 5kB to unpack).
Bitbreaker's optimization on his bitfire loader helped a lot and let me choose even the wildest part of 2 seconds in the games.
But, as Burglar states, I did not implement any support for large area scrolling, so the games could not be full screen scrollers.
Also, bitmap mode graphics would require too much space on disk, so I skipped those.
And, for games with builtin sprite multiplexers, I would need to choose the position on the screen where the most interesting sprite graphics were. I had to give up on Green Beret and one Commando scene due to this.
I do not support graphics mode switching or raster interrupts or bank switching, so the upper half of the Ghettoblaster screen had to be empty.
In some games I had to remove some of the rows of the graphics to save diskspace. Bitbreaker's Geos-part that ends the disk uses 7,5kB, so some static texts with score had to go. Also, the "Another visitor. Stay a while. Stay forever"-sample is actually loaded before the noisefader part starts on disk #1, and is kept in memory and used five minutes later in the demo.

2023-06-23 05:26

Bansai

Registered: Feb 2023
Posts: 35

Quoting Raistlin

Pex's answer:-
This leads to a 100 to 200 lists of memory locations with 100 values that needs to be written to replay the graphics and sound.

For those who deal with Verilog/VHDL simulator design, that's exactly how many modern sim dump formats store the staggering amounts of signal data generated in sim, yet make it compressible and quickly accessible for waveform viewer tools. As time goes on, I notice more and more parallels between 8-bit assembly language programming and logic design work.

2023-06-23 10:59

Bitbreaker

Registered: Oct 2002
Posts: 501

Quoting Pex Mahoney Tufvesson

The largest games on disk are Bombjack (3167 bytes), Giana Sisters and Revenge of the mutant camels, all larger than 3kB...

In sum there's 807kb of data prior to being packed with dali. The packed data then pretty much fills up the whole disk. So also zx0 compression helps a lot and that would not have fit with the old Doynax/Bitnax or sectorwise compression mechanisms.

2023-06-23 11:40

Krill

Registered: Apr 2002
Posts: 2855

Quoting Bitbreaker

In sum there's 807kb of data prior to being packed with dali. The packed data then pretty much fills up the whole disk. So also zx0 compression helps a lot and that would not have fit with the old Doynax/Bitnax or sectorwise compression mechanisms.

So, 100 tiny files totalling to 807 KB are compressed to about 170 KB on disk, without the block allocation overhead of 127 bytes per file on average.

Out of some interest, how much smaller is the packed data if you compress all 100 tiny files as a single blob? =)
(Obviously, i'm aiming at stream decrunching as a size optimisation opportunity.)

2023-06-23 12:05

Bitbreaker

Registered: Oct 2002
Posts: 501

The 100 games pack to 149799 bytes, the raw data glued all together to a single blob pack to 145904 bytes.
How are you resolving large offsets of up to 15 bit with streaming the data? You still need old data to be there in RAM to be referenced?
Can provide the raw data if wanted.

2023-06-23 12:11

Krill

Registered: Apr 2002
Posts: 2855

Quoting Bitbreaker

The 100 games pack to 149799 bytes, the raw data glued all together to a single blob pack to 145904 bytes.

Okay, that's about 16 blocks saved. Not much. =| But about one track on the inner rim. =D

Quoting Bitbreaker

How are you resolving large offsets of up to 15 bit with streaming the data? You still need old data to be there in RAM to be referenced?

You limit the offsets to however big the streaming ringbuffer is - backrefs never fall out of the sliding window. Have a look at Exomizer's exostreamdecr.s for an example. :)

For practical purposes in a demo, though, you'd just split up the blob to approximately equal-size packed chunks, then load into alternating slots.
It's not as fancy as true stream decrunching, though.

2023-06-23 13:18

Bitbreaker

Registered: Oct 2002
Posts: 501

Sure, that sliding window thing also was implemented in doynax packer back then, it limits the maximum offset, depending on how big the buffer with sliding window will be. So you loose again some pack ratio by that, as soon as you go below 32kb. Worst case is a 256 byte sector buffer. And then again, you melt it down to two buffers to depack to alternatingly, if done right, one can at least reference one of the preceeding data as dict if it is within offset range. So overall it is mostly about convenience that is traded with memory footprint for a bigger buffer compared to just loading alternatingly, what is not much effort codingwise as well. Another advantage of loading alternatingly: You can directly load/depack to the right destination and have the initial charset ready without any additional copy action (and memory) from a buffer.

2023-06-23 13:37

Krill

Registered: Apr 2002
Posts: 2855

Quoting Bitbreaker

Another advantage of loading alternatingly: You can directly load/depack to the right destination and have the initial charset ready without any additional copy action (and memory) from a buffer.

True stream decrunching doesn't make so much sense when you need to copy from the stream buffer.

If i understood Pex's approach right, the stream's poke-data could be read directly and applied, without any additional copying from the stream buffer. Initial charsets etc. would still need to be copied around, though, unless you format the depacked data just right so the charsets sit at the correct position. Very hacky, though. =)

But yeah, depending on how big or not so big the streaming buffer is, pack ratio suffers, eating into the gain of crunching the entire blob.

Previous - 1 | 2 | 3 | 4 - Next

Refresh

Subscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search for in
All times are CET.

Search CSDb

Advanced

Users Online

Exploding Fi../Techn..
saimo/RETREAM
Magnar
Freeze/Blazon
Knut Clausen/SHAPE/F..
Frost/Triad
Wiklund/Fairlight
Guests online: 99

Top Demos

1 Next Level  (9.8)
2 13:37  (9.7)
3 Mojo  (9.7)
4 Aliens in Wonderland  (9.7)
5 Coma Light 13  (9.7)
6 Edge of Disgrace  (9.6)
7 No Bounds  (9.6)
8 Comaland 100%  (9.6)
9 Uncensored  (9.6)
10 Wonderland XIV  (9.6)

Top onefile Demos

1 Happy Birthday Dr.J  (9.8)
2 Layers  (9.6)
3 It's More Fun to Com..  (9.6)
4 Cubic Dream  (9.6)
5 Party Elk 2  (9.6)
6 Copper Booze  (9.6)
7 TRSAC, Gabber & Pebe..  (9.5)
8 Rainbow Connection  (9.5)
9 Dawnfall V1.1  (9.5)
10 Daah, Those Acid Pil..  (9.5)

Top Groups

1 Nostalgia  (9.4)
2 Oxyron  (9.3)
3 Booze Design  (9.3)
4 Censor Design  (9.3)
5 SHAPE  (9.3)

Top Diskmag Editors

1 Jazzcat  (9.4)
2 Magic  (9.4)
3 hedning  (9.2)
4 Elwix  (9.1)
5 A Life in Hell  (9.1)

Page generated in: 0.04 sec.