Disk usage - read for your own education

[QUOTE="lordxymor"][QUOTE="Mortok"]

PC games span accross multiple CDs, not DVDs or Blueray, the gameplay is still enjoyable, Graphics is still beautiful and the story is still immersive. Your arguement has no bearing on this topic, the TC isn't attacking any platform but he is explaining the misconception of multi-disk games. He's explaining what PC gamers have known for awhile.

World of Warcraft, Company of Heroes, World in Conflict, Unreal Tournament 3 and Crysis all make Killzone 2's 2 gig level design .doc seem like trans fat -- neither required nor beneficial.

Redfingers

But PC games are installed, when they are decompressed. If you compress a console game in the same levels of PC games, you're gonna waste mad ammounts of memory and cpu to decompress everything in real-time.

Here's where the argument gets interesting. Warhawk takes up 800 MB cache extremely compressed on the hard drive, both the retail and downloadable versions.

The developer said in an interview with IGN that this 800 megs stays 800 megs until the very moment you decide to demand the content, at which point it sits on main memory. Main memory meaning 256 MB XDR memory, I presume. How this works I do not know but the game works without a single hitch and it does indeed work.

It's a common misconception that compression takes huge amounts of memory + CPU power. It takes some, but the slowness you see in programs like Winzip are disk-bound, not CPU-bound. (i.e., it's slow because it's writing to disk) Furthermore, certain things are more compressable than others. A quick compression lesson:

Suppose your data is 11110000. If your compression scheme is 1 = 1111 and 0 = 0000, you can write "10" on your disk and be done.

Suppose your data is now 10101110. It's not as uniform, so you might do something like 1 = 10, 0 = 11 and you'd have to write "1101" on your disk to make it work.

With millions of bytes, there are algorithms that approximate the best schemes for the data, but you get the idea. In general, media is highly compressible (even lossless, and we can go really far w/ lossy compression), level information is also high compressible (think of points of a polygon, a lot of coordinates in a real game "line up"), but code isn't.

This is meaningless. To a map designer, the term "level data" is defined as geometry and actor/mesh/texture placement data. The resources themselves..meaning texture data, models, etc are effectively part of the game itself, and can not attributed to any given level (since they're recycled throughout.) A developer would have to be outright LYING to say that data alone takes up anywhere near 2GB. It simply isn't possible unless they're idiotically packaging texture data and other resources with map data, which is a totally ineffective way of going about it. It's MANY times more effective to pull level resources from one single pool.

It sounds like they might be using a stupid and ineffective method of resource pooling to try and justify Blu-Ray's usage. I really doubt that's the case, but it's either that or they aren't using the term "level data" properly (this is most likely.) Or they're lying, take your pick.Velocitas8

There are perfectly legitimate reasons for storing levels as a whole unto themselves on a disk. R:FoM showed that it's simply more efficient to read off of the BluRay continuously instead of repeatedly seeking + reading data, and in this case the size of the media compensates for its latent access. Console development is different from PC development precisely because of this reason - developers work with lesser but standardized hardware and they're able to get more out of it, special instances that run contrary to the "standard case".

[QUOTE="Pro_wrestler"]:) Mass Effect is a perfect example how taxing the game is to the CPU with its 100's of different locations and cutscenes rendered in real-time.[QUOTE="lordxymor"]

I like to see how BioWare strategy with Mass Effect.

That's the type of game that would really benefit from blu-ray, but they managed to squeeze it into just 7GB. They're probably making full use of procedural synthesis to save up textures space and no pre-rendered video whatsoever but no matter how highly effcient engine is, there's one fundamental thing they can't optimize, audio.

High quality audio takes massive ammounts of space and considering this is an RPG with lots of dialogs, I ask myself if most of the game conversations won't be text reading instead of the character actually speaking.

TC is very right, blu-ray is an advantage, specially in games like MGS, ME and GTA, but nothing stops developers from making fun and great looking games with dvd9 space or less.

Redfingers

Uhh..Compression? WMA is almost twice the cize of MP3's yet the difference is unnoticeable.

I've seen the videos of Mass Effect and absolutely everything involves voice acting. It being poorly acted (which I believe a lot of it probably is) is irrelevant.

I wasn't refering to its voice acting. I was talking about things being rendered in real-time rather than CG. Same with ES4, It had basically 1 CG sequence and that was the opening pan of the Imperial City. Gears is an example of real-time sequences.

There are perfectly legitimate reasons for storing levels as a whole unto themselves on a disk. R:FoM showed that it's simply more efficient to read off of the BluRay continuously instead of repeatedly seeking + reading data, and in this case the size of the media compensates for its latent access.Datheron

I see, that sounds reasonable. Does it really make that much of a difference, though? I'm not terribly familiar with optical storage, but does sector seek-time really make up that much of a game's loading time? Either way I suppose any bit of benefit would be enough to justify its usage, since Blu-Ray has massive amounts of storage space to throw around anyways.

And wouldn't the HDD caching of game resources (since the HDD is standard on the PS3) help significantly cut down on seeking and loading times, too? Though obviously you have to move the resources from the disc to the cache in the first place in order to see any benefit.

Very informative read, thank you.

Now maybe some people will realize that blu ray really will not always be utilized as much for the length of a game, but instead it will be utilized to be able to have more individual variety in enviroments, effects, ect. Such as what Naughty Dog is doing with Uncharted. It may only be a 12 hour game, but the blu ray disk space is not being used for game length. It is being used for all of the individual details that they are putting into the game, from the enviroments, animations, physics and so on.

Alot of people will look at that game and say, "it can be done on DVD9 as well." Sure it could, but it would not be able to hold all of the individual, independent enviroment details, effects, ect on one DVD9 disk that they are packing into the blu ray disk. And that is why they said that blu ray was absolutely needed for this game the way that they are making it. Game length has nothing to do with it.

Isn't the biggest issue with the PS3 the split memory?

No matter how big the game they can only fit 256MB of stuff into RAM at one time.

There is no way around that.

Just read a few threads on the recent Lost Odyessy 4-disc fiasco, and I gotta say that as a software developer, it's sad to see various pieces of misinformation about how hardware is used and have that kind of misunderstanding repeated over and over again until it becomes some sort of accepted truth.

I'll try to explain what is really used for disk space (versus internal system memory and GPU memory), hopefully so that the clearly wrong and tired arguments are put to rest. As to whether people personally care about swapping discs, I'll leave that up to the current threads.

• Disk space is used mainly for media: pre-rendered video, audio, and textures. Note that the elements of in-game cutscenes take up space - stuff like characters speaking and new objects which can only be seen in that cutscene - but the animation itself and the movements are actually miniscule in size. For example, if you use existing game polygonal models and have characters talk to each other, the scene is going to be rather cheap in terms of diskspace. If your scene has explosions and you have to change a "clean" model of a building to a "destroyed" model, that's going to take up space.
• "Gameplay" - the programming code that defines what you can/cannot do in a game, actually uses very little disk space by itself. Consider that the 100 million+ lines of Windows Vista code can be compiled into 100-200MB's of executable binaries, e.g., runnable code. Also consider how much gameplay is had for 1-2 MB's in the original Civilization and Simcity.
• By extension, the length of a game is not at all dependent on how much disk space is available. FFVI was 40-60 hours long and came on a 16MB cart, and of course Oblivion fits on a DVD9. What disk space buys you is the ability to have lots of different premade objects in your game world, such as a forest that has 200 different kinds of trees versus one tree repeated 200 times. (of course, it's unrealistic for a game dev. to spend time creating 200 different trees just b/c he has the disk space to do so) Games like Oblivion and even Diablo II get away with this because they procedurally generate their maps so it taxes the CPU instead; it wouldn't work for something like GTAIV which tries to model a highly recognizable city.
• For most games on mult. disks, a good chunk of the space on disk is the same; for example, for multi-disk RPG's stuff like the battle system, character models, overworld map, etc. are going to have to be copied on every disk because the player can control these things. In that sense, the reason for multiple disks really is for the cinematics, but it's not a simple 3 disks * 9 GB = 27GB of data.

Ultimately, think of disk space as a way to save computational and rendering time. Having very high resolution texture of a chain-link fence means you don't have to spend the resources to render the thousands of polygons that make up each chain of that fence. Putting in a pre-rendered movie means you don't have to figure out how to animate your in-game characters within the limitations of the game engine (e.g., look at the limited range of animations for characters in FFVII in the game world). Having two models of buildings lit differently means you don't need a complex (and computationally expensive) lighting model on buildings to get day/night graphics.

There's always going to be a compromise between CPU and disk space, and there's no hard-and-fast rule dictating whether having more disk space is good or not.

Datheron

It is true that diskspace is mainly used for media such as videos, sounds, and textures. Same for code.

There are couple of things you have not touched and not everything is correct. For example, you stated about models taking up space, they dont. It's the textures you use on them that do. Also using very high resolution textures is pretty bad too.

Just because you have 50gb of storage space, doesn't mean you can use them uncompressed everywhere. Remember, your video buffer might be limited to 128 to 256mb of video ram reserved depending on the system you develope. More true for the consoles than on PC. You DON'T want to go over that because now you'll be dealing with virtual memory reserved and another bottleneck to deal with which could sacrifice your performance. Also with higher resolution textures, it stresses the GPU more now because it has more data to compute and render. I always feel that a higher polygon count is worth more than a better texture because it will seem more fluid from an artistic point of view. However, one thing I notice alot about next gen games is that they dump all the high poly and textures on models and yet the environments look like something from 5 years ago.

I remember when I worked on Transport Giant for the PC. Our maximum buffer for graphics was 90mb. We filled that up within couple of hours and it only contained about maybe 1/20th of the game. So we made pretty early screenshots using those. Of course we had to start compressing and lowering the quality to fit everything into that 90mb limit. Do note, this is for the PC and that time we were aiming for 128mb video card recomendations.

edit: also one thing I forgot to mention, when working on a PC, all the game components were installed on the harddrive so there was nothing needed to read from the CD except for a cd-check script right at the beggining when launching the game. For consoles, the speed and seek-time of the drive will make a HUGE difference in terms of loading the game information into memory. Having an active stream from a media source is also a bad programming decision so it's best to make sure that all your gama data can fit into ram and virtual memory.

Isn't the biggest issue with the PS3 the split memory?

No matter how big the game they can only fit 256MB of stuff into RAM at one time.

There is no way around that.

BlueBarad

This is true, but as far as game length goes, it is not going to be an issue. As how many games are going to come out this gen that are going to be so many hours that it simply could not fit, lol? As the TC said, FFI back in the day was 40 hours, so I think the PS3 will be fine in terms of being able to run"big" games, lol.

And blu ray will allow for more intricate work to be done within the game that cannot be done on DVD9. This is just a fact, as it has more space to be utilized for such. Game length has nothing to do with it. So why people bash blu ray when it actually does allow for more intricate game developing is beyond me.

[QUOTE="Redfingers"]The entirely of a PC game is installed to the hard disc, meaning the TC's point about LO being primarily recycled material is completely and utterly irrelevant on the PC platform.Velocitas8

Game materials aren't recycled due to disc space limitations as you ignorantly imply. Resources are recycled due to SYSTEM MEMORY limitations, which Blu-Ray doesn't assist with easing IN THE LEAST. Another reason is it requires alot less work and therefore saves money to recycle the same authored content throughout any given level.

Blu-Ray is absolutely meaningless in the pursuit of utilizing unique resources throughout a game. Disc space has never been the issue with unique textures and other game resources, it's always been RAM. Especially with consoles, which are extremely memory-bound. Even then, those limitations can be overcome with procedural generation.

he fact that a level in Killzone 2 takes up 2 gigabytes, according to the developer, is EXTREMELY significant, not insignificant in the face of what these other games constitute.Redfingers

This is meaningless. To a map designer, the term "level data" is defined as geometry and actor/mesh/texture placement data. The resources themselves..meaning texture data, models, etc are effectively part of the game itself, and can not attributed to any given level (since they're recycled throughout.) A developer would have to be outright LYING to say that data alone takes up anywhere near 2GB. It simply isn't possible unless they're idiotically packaging texture data and other resources with map data, which is a totally ineffective way of going about it. It's MANY times more effective to pull level resources from one single pool.

It sounds like they might be using a stupid and ineffective method of resource pooling to try and justify Blu-Ray's usage. I really doubt that's the case, but it's either that or they aren't using the term "level data" properly (this is most likely.) Or they're lying, take your pick.

Point 1: you totally misinterpreted what I was talking about, reread the post and its context.

I wasn't talking about recycled content on one single disc, I was referring to what the TC said about content that was necessary (character models and essential info and whatnot....everything that is static from one disc to the next) being copied, therefore not taking up the full 9 gigs in 100% progressive game material.

Seeing as disc 1 does not equal disc 2 it's a totally valid assumption that system memory has absolutely nothing to do with this particular kind of repetition. However, I do appreciate you calling me ignorant, and I hope you won't dispute the irony now that everything's been cleared up.

Question: Have you seen the first level in Killzone? Would you like to? Well, here you go:

http://kotaku.com/gaming/gc07/killzone-2-developer-walkthrough-293604.php

I'm going to go ahead and guess that you've never seen that before.

[QUOTE="BlueBarad"]

Isn't the biggest issue with the PS3 the split memory?

No matter how big the game they can only fit 256MB of stuff into RAM at one time.

There is no way around that.

ironcreed

This is true, but as far as game length goes, it is not going to be an issue. As how many games are going to come out this gen that are going to be so many hours that it simply could not fit, lol? As the TC said, FFI back in the day was 40 hours, so I think the PS3 will be fine in terms of being able to run"big" games, lol.

And blu ray will allow for more intricate work to be done within the game that cannot be done on DVD9. This is just a fact, as it has more space to be utilized for such. Game length has nothing to do with it. So why people bash blu ray when it actually does allow for more intricate game developing is beyond me.

What kind of "more intricate work" do you mean?

On the coding side? Particle effects? Physics? Bump mapping?

• FFVI was 40-60 hours long and came on a 16MB cart, and of course Oblivion fits on a DVD9.

Datheron

Adding to that - Morrowind had a hell of a lot more gameplay content than Oblivion (eg. you could easily do everything in Oblivion in under 70 hours, but Morrowind can take over a hundred hours), and that fit on a single CD.

Then again, Daggerfall was even larger, and I highly doubt that that filled a whole CD...

[QUOTE="BlueBarad"]

Isn't the biggest issue with the PS3 the split memory?

No matter how big the game they can only fit 256MB of stuff into RAM at one time.

There is no way around that.

ironcreed

This is true, but as far as game length goes, it is not going to be an issue. As how many games are going to come out this gen that are going to be so many hours that it simply could not fit, lol? As the TC said, FFI back in the day was 40 hours, so I think the PS3 will be fine in terms of being able to run"big" games, lol.

And blu ray will allow for more intricate work to be done within the game that cannot be done on DVD9. This is just a fact, as it has more space to be utilized for such. Game length has nothing to do with it. So why people bash blu ray when it actually does allow for more intricate game developing is beyond me.

Misconception FTW: As I understand it, both the video and main memory in the Playstation 3 are faster than the memory in the Xbox 360. The main memory is higher bandwidth, lower latency, and the video memory is faster due to being dedicated video memory as opposed to shared memory.

The problem, and this is the problem that Carmack was referring to, FYI, is that the Playstation 3 reserves significantly more memory for the operating system than the Xbox 360. However, Carmack's figures were incorrect. The number is being dramatically reduced and should be offset by the speed of the memory itself...at least partially.

Additionally, as I understand it, the memory in the Playstation 3 is, while partitioned, able to be accessed by both the GPU and the CPU (referring to both main and video memory). Read this Sony report in which Sony apparently has an identity crisis and refers to the Playstation 3 as having "unified memory." Yes, the Playstation 3 can access both pools of memory despite the partition, even performing an app simultaneously:

http://research.scea.com/ps3_deferred_shading.pdf

"The system's unified memory architecture allows
the Cell/B.E. and GPU to exchange data through shared textures."

Yes, that says unified memory, and exchange data through shared textures.

Additionally, as I understand it, the Xbox 360 actually partitions the memory...how that memory is partitioned is up to the developer, but once it begins an app, again, as I understand it (correct me if I'm wrong), the memory is partitioned. You don't have 512 video memory and 512 system memory available for the choosing, you have the partitions that were determined by the developer. The developers looking at a unified 512 MB (minus 32 for OS...less than PS3, remember), but he can't pull additional memory out of thin air. The question is efficiency, and that's the question the Xbox 360 answers, but with regards to performance, it is not tremendously more flexible, and certainly not more capable.

[QUOTE="Datheron"]

• FFVI was 40-60 hours long and came on a 16MB cart, and of course Oblivion fits on a DVD9.

Planeforger

Adding to that - Morrowind had a hell of a lot more gameplay content than Oblivion (eg. you could easily do everything in Oblivion in under 70 hours, but Morrowind can take over a hundred hours), and that fit on a single CD.

Then again, Daggerfall was even larger, and I highly doubt that that filled a whole CD...

Oh, no, my world is crumbling around me! Why can't we play 8-bit games again?

[QUOTE="ironcreed"][QUOTE="BlueBarad"]

Isn't the biggest issue with the PS3 the split memory?

No matter how big the game they can only fit 256MB of stuff into RAM at one time.

There is no way around that.

BlueBarad

This is true, but as far as game length goes, it is not going to be an issue. As how many games are going to come out this gen that are going to be so many hours that it simply could not fit, lol? As the TC said, FFI back in the day was 40 hours, so I think the PS3 will be fine in terms of being able to run"big" games, lol.

And blu ray will allow for more intricate work to be done within the game that cannot be done on DVD9. This is just a fact, as it has more space to be utilized for such. Game length has nothing to do with it. So why people bash blu ray when it actually does allow for more intricate game developing is beyond me.

What kind of "more intricate work" do you mean?

On the coding side? Particle effects? Physics? Bump mapping?

Naughty Dog said the streaming was only possible through Blu Ray and the HDD. Don't ask me how. Write them a letter.

[QUOTE="Redfingers"][QUOTE="Mortok"][QUOTE="Redfingers"][QUOTE="BuryMe"]

Excellent post.

People need to realise that gaming can be just as good on a 16mb cart as it can be on a 25 gig blu-ray disc. as long as developers are creative with how they use the space they have, any system can give us long, interesting games.

Mortok

Then guess what: go play games on a Super Nintendo and leave the rest of us alone.

You forfeit your argument.

Bitter?

PC games span accross multiple CDs, not DVDs or Blueray, the gameplay is still enjoyable, Graphics is still beautiful and the story is still immersive. Your arguement has no bearing on this topic, the TC isn't attacking any platform but he is explaining the misconception of multi-disk games. He's explaining what PC gamers have known for awhile.

World of Warcraft, Company of Heroes, World in Conflict, Unreal Tournament 3 and Crysis all make Killzone 2's 2 gig level design .doc seem like trans fat -- neither required nor beneficial.

The entirely of a PC game is installed to the hard disc, meaning the TC's point about LO being primarily recycled material is completely and utterly irrelevant on the PC platform. Notice: no one's arguing the viability of Blu Ray ATM on the PC platform...if ever. Hard drives are essentially limitless in capacity but this is NOT VIABLE in the console market.

You're the one with an incongruity to this topic. I don't believe PC games were ever addressed nor were their relevance ever implied anywhere.

Let's evaluate your glib "bitter?" comment in context. You asked if I was bitter when I shut down someone's argument for a point they mentioned that was relevant to the topic. No sweat. You, on the other hand, brought up something that was completely irrelevant to either my argument, my post, or the thread itself and then proceeded to ask me if I was bitter about something I had already shot out of the sky. If you're NOT bitter, I ask you, why did you reply to that post and not even address my actual argument?

Does the PC use 16 MB cartridges? Does the Super Nintendo have a hard drive? Just stop harrassing me with your nonsensical insults.

World of Warcraft is crap, Crysis isn't out yet, UT3 isn't out yet, World in Conflict I doubt you've even played (or purchased, yet), assuming it's out, and Company of Heroes is an awesome game. However, it doesn't take anywhere near 2 gigabytes of space per level. You're totally forgetting the relevance of the argument. The fact that a level in Killzone 2 takes up 2 gigabytes, according to the developer, is EXTREMELY significant, not insignificant in the face of what these other games constitute.

PC platform, NES, what is the difference here? They are both mediums that do not use blueray nor need to. First, I wasn't attacking you. You were harsh on Buryme when he made his opinion. Second,when you are paraphrasing a developer in their reason why 2 gigs is needed when we know what was said with Lair, we know was said with Heavenly Sword, really mean nothing. nothing.

The NES doesn't need Blu Ray, but the NES is not a PC, nor is it a Playstation 3 nor an Xbox 360.

That's basic, and that's my underlying point. Additionally, refer to the PC, NES, etc, as platforms and storage media as medium.

And Buryme can go play Super Nintendo any day of the week if he wants but excuse me if I take it for granted that that won't get him Brownie points as far as a legitimate argument goes.

With Heavenly Sword, we know that audio was probably the largest segment of data consuming space on the Blu Ray disc....they had a lot of extremely high quality audio they wanted to preserve from high quality actors....probably followed by motion capture.

I think it's pretty juvenile to assume that all of this stuff is just fluff, though. On one end of the spectrum you have these guys absolutely filling up discs, and on the other end, you have people conservatively pushing and compressing and poking and prodding....so what do you think? I take the high road and assume that Lair was filled with gigabytes of legitimate information, whether or not it was a good game. You cannot ask me to figuratively compress the 25 gigs into a comparative experience of 7 gigabytes or something like that....the value of a game is not determined by the space it consumes. That's Kidnergarten stuff. We're not arguing that. We're arguing whether the space was taken up by legitimate information, and just looking at the game and all the stuff its got in it would tell you that.

They have orchestral scores, all in 7.1, mixed animal sounds and high quality recreations, huge, expansive environments, lots of creatures, animations, etcetera. Once I remember Julian Eggebrecht explaining all of the stuff that they pack in there....he didn't say "audio, audio, audio," and I suppose you could refer to that as filler, but the rest of it is pretty damn legit and you'd be hard pressed to tell me they could fit half of that game on a DVD-9 if they...y'know, squeezed?

Plus, they had a bunch of CG. I know what went into the game, thank you.

[QUOTE="ironcreed"][QUOTE="BlueBarad"]

Isn't the biggest issue with the PS3 the split memory?

No matter how big the game they can only fit 256MB of stuff into RAM at one time.

There is no way around that.

BlueBarad

This is true, but as far as game length goes, it is not going to be an issue. As how many games are going to come out this gen that are going to be so many hours that it simply could not fit, lol? As the TC said, FFI back in the day was 40 hours, so I think the PS3 will be fine in terms of being able to run"big" games, lol.

And blu ray will allow for more intricate work to be done within the game that cannot be done on DVD9. This is just a fact, as it has more space to be utilized for such. Game length has nothing to do with it. So why people bash blu ray when it actually does allow for more intricate game developing is beyond me.

What kind of "more intricate work" do you mean?

On the coding side? Particle effects? Physics? Bump mapping?

Well, I am no uber tech head, but it is not rocket science here, lol. Blu Ray offers more storage space, and since game length is not what takes up most of the space on the disk, there is considerably more space than DVD9 to concentrate on creating levels with more individual effects, rather than having repeating models throughout a game, such as in Oblivion's case with the trees, characters, voices, dungeons, temples and so forth. Or in Dead Risings case with the same models of zombies used in every area.

The larger disk space allows for more lighting effects, more animations, characters with individual physics, and things of this nature. Quite simply, there is just more freedom, and literally more room to go into higher detail in making the game, as the disk space allows for it. And that is why devs like Naughty Dog said that with the amounts of detail that they have put into Uncharted, they had to have blu ray in order to do it the way they are making it. Game length has nothing to do with it.

[QUOTE="Redfingers"][QUOTE="thirstychainsaw"][QUOTE="Redfingers"]

Silence, Guerilla Games is speaking. Let's listen in:

"One level in Killzone 2 takes up 2 gigabytes."

Datheron

Wasn't the same things said about Lair and Heavenly Sword?

No. I addressed Heavenly Sword in my earlier post.

Heavenly Sword takes up space by using assets that DON'T have to do with gameplay/level data/graphics. Rather, it's cinematic/animation/voice acting/audio based.

The point is that a LEVEL, I.E. an actual segment of the game in Killzone 2 takes up 2 gigabytes and that's inarguable. That isn't fluff. That isn't erroneous crap that you could "live without," that's gameplay. Not game CODE, persay, but then again, what would you do with game code if you didn't have a level? Feet without a playground...

Lemme clear this up. This, along with the "300MB" levels in Unreal Tournament, refers to the entirety of the level, including the textures, the surface definitions, the layout, and ultimately each of the points on the polygons that define that burning car you see down the street.

Now, if you have 20 levels, there's a few days you can go about serializing the information. (e.g., taking raw data and putting it on disk) Each level can embed everything it needs unto itself and be one gigantic file (or set of files) - that's where that 2GB comes in. Or, you have a set of common resources - say, a directory full of textures and objects - and you reference these resources from your map; that'd bring down your map definition information to a few MB's as you're mostly grabbing from a common pool of data. That common pool, however, may be many GB's in size.

Let me preface by saying that Guerilla followed the "2 gigabyte" comment by saying that the game will absolutely not fit on anything less than a Blu Ray disc. The only legitimate argument against this is that, essentially, he's either lying or paid by Sony to say that...

I'm just not certain he's bloating the disc up with arbitrary information. He said, "yes, the level is 2 gigabytes, and, it follows that we cannot fit the game on a DVD-9." That assumes there's plenty where that came from, be it textures, layout, etcetera.

I really cannot force myself to believe even for a moment that that figure is erroneous.

[QUOTE="ironcreed"][QUOTE="BlueBarad"]

Isn't the biggest issue with the PS3 the split memory?

No matter how big the game they can only fit 256MB of stuff into RAM at one time.

There is no way around that.

Redfingers

This is true, but as far as game length goes, it is not going to be an issue. As how many games are going to come out this gen that are going to be so many hours that it simply could not fit, lol? As the TC said, FFI back in the day was 40 hours, so I think the PS3 will be fine in terms of being able to run"big" games, lol.

And blu ray will allow for more intricate work to be done within the game that cannot be done on DVD9. This is just a fact, as it has more space to be utilized for such. Game length has nothing to do with it. So why people bash blu ray when it actually does allow for more intricate game developing is beyond me.

Misconception FTW: As I understand it, both the video and main memory in the Playstation 3 are faster than the memory in the Xbox 360. The main memory is higher bandwidth, lower latency, and the video memory is faster due to being dedicated video memory as opposed to shared memory.

The problem, and this is the problem that Carmack was referring to, FYI, is that the Playstation 3 reserves significantly more memory for the operating system than the Xbox 360. However, Carmack's figures were incorrect. The number is being dramatically reduced and should be offset by the speed of the memory itself...at least partially.

Additionally, as I understand it, the memory in the Playstation 3 is, while partitioned, able to be accessed by both the GPU and the CPU (referring to both main and video memory). Read this Sony report in which Sony apparently has an identity crisis and refers to the Playstation 3 as having "unified memory." Yes, the Playstation 3 can access both pools of memory despite the partition, even performing an app simultaneously:

http://research.scea.com/ps3_deferred_shading.pdf

"The system's unified memory architecture allows
the Cell/B.E. and GPU to exchange data through shared textures."

Yes, that says unified memory, and exchange data through shared textures.

Additionally, as I understand it, the Xbox 360 actually partitions the memory...how that memory is partitioned is up to the developer, but once it begins an app, again, as I understand it (correct me if I'm wrong), the memory is partitioned. You don't have 512 video memory and 512 system memory available for the choosing, you have the partitions that were determined by the developer. The developers looking at a unified 512 MB (minus 32 for OS...less than PS3, remember), but he can't pull additional memory out of thin air. The question is efficiency, and that's the question the Xbox 360 answers, but with regards to performance, it is not tremendously more flexible, and certainly not more capable.

So the PS3 is not going to be crippled by it's split memory after all because it shares the load between the Cell/B.E. and GPU, correct? That is how I am understanding it in laymans terms at least., lol.

[QUOTE="Datheron"]There are perfectly legitimate reasons for storing levels as a whole unto themselves on a disk. R:FoM showed that it's simply more efficient to read off of the BluRay continuously instead of repeatedly seeking + reading data, and in this case the size of the media compensates for its latent access.Velocitas8

I see, that sounds reasonable. Does it really make that much of a difference, though? I'm not terribly familiar with optical storage, but does sector seek-time really make up that much of a game's loading time? Either way I suppose any bit of benefit would be enough to justify its usage, since Blu-Ray has massive amounts of storage space to throw around anyways.

And wouldn't the HDD caching of game resources (since the HDD is standard on the PS3) help significantly cut down on seeking and loading times, too? Though obviously you have to move the resources from the disc to the cache in the first place in order to see any benefit.

That would probably be the reason that Naughty Dog said that their streaming data simultaneously from HDD and Blu Ray allowed for 0 loading times.

[QUOTE="Redfingers"][QUOTE="ironcreed"][QUOTE="BlueBarad"]

Isn't the biggest issue with the PS3 the split memory?

No matter how big the game they can only fit 256MB of stuff into RAM at one time.

There is no way around that.

ironcreed

This is true, but as far as game length goes, it is not going to be an issue. As how many games are going to come out this gen that are going to be so many hours that it simply could not fit, lol? As the TC said, FFI back in the day was 40 hours, so I think the PS3 will be fine in terms of being able to run"big" games, lol.

And blu ray will allow for more intricate work to be done within the game that cannot be done on DVD9. This is just a fact, as it has more space to be utilized for such. Game length has nothing to do with it. So why people bash blu ray when it actually does allow for more intricate game developing is beyond me.

Misconception FTW: As I understand it, both the video and main memory in the Playstation 3 are faster than the memory in the Xbox 360. The main memory is higher bandwidth, lower latency, and the video memory is faster due to being dedicated video memory as opposed to shared memory.

The problem, and this is the problem that Carmack was referring to, FYI, is that the Playstation 3 reserves significantly more memory for the operating system than the Xbox 360. However, Carmack's figures were incorrect. The number is being dramatically reduced and should be offset by the speed of the memory itself...at least partially.

Additionally, as I understand it, the memory in the Playstation 3 is, while partitioned, able to be accessed by both the GPU and the CPU (referring to both main and video memory). Read this Sony report in which Sony apparently has an identity crisis and refers to the Playstation 3 as having "unified memory." Yes, the Playstation 3 can access both pools of memory despite the partition, even performing an app simultaneously:

http://research.scea.com/ps3_deferred_shading.pdf

"The system's unified memory architecture allows
the Cell/B.E. and GPU to exchange data through shared textures."

Yes, that says unified memory, and exchange data through shared textures.

Additionally, as I understand it, the Xbox 360 actually partitions the memory...how that memory is partitioned is up to the developer, but once it begins an app, again, as I understand it (correct me if I'm wrong), the memory is partitioned. You don't have 512 video memory and 512 system memory available for the choosing, you have the partitions that were determined by the developer. The developers looking at a unified 512 MB (minus 32 for OS...less than PS3, remember), but he can't pull additional memory out of thin air. The question is efficiency, and that's the question the Xbox 360 answers, but with regards to performance, it is not tremendously more flexible, and certainly not more capable.

So the PS3 is not going to be crippled by it's split memory after all because it shares the load between the Cell/B.E. and GPU, correct? That is how I am understanding it in laymans terms at least., lol.

Supposedly there's a performance hit, but these particular Sony employees didn't seem to see any issues...and given they refer to the platform as having a unified memory architecture I can't see why not, unless you doubt their legitimacy, and I do not.

Teufelhun, a System Wars tech guy told me that the performance hit is significantly harder on the video memory than the main memory, so perhaps that's what they're sharing. However, the PDF article specifically refers to sharing textures....

I'd say it's possible but I'd like independent confirmation.

[QUOTE="ironcreed"][QUOTE="Redfingers"]

Redfingers

So the PS3 is not going to be crippled by it's split memory after all because it shares the load between the Cell/B.E. and GPU, correct? That is how I am understanding it in laymans terms at least., lol.

Supposedly there's a performance hit, but these particular Sony employees didn't seem to see any issues...and given they refer to the platform as having a unified memory architecture I can't see why not, unless you doubt their legitimacy, and I do not.

Teufelhun, a System Wars tech guy told me that the performance hit is significantly harder on the video memory than the main memory, so perhaps that's what they're sharing. However, the PDF article specifically refers to sharing textures....

I'd say it's possible but I'd like independent confirmation.

You know, I would like to think that through all of the R & D that they would not just overlook such a limiting hit due to split memory. Beings that they named it "Unified memory architecture", I would surmise it is because that is indeed the case, which I interpret as the hit not being signifigant enough to warrant any cause for alarm as far as any major performance issues go. I just cannot see with all that went into the PS3, that they would leave it crippled straight out of the gate.

Iron, notice how Redfinger's "studies" are all from Sony.

The 360 was crippled also with only 256MB of RAM, until Epic showed M$ Gears of War with and without the extra 256MB of RAM.

M$ decided to take a 1 billion loss right there to put the extra memory.

Iron, notice how Redfinger's "studies" are all from Sony.

The 360 was crippled also with only 256MB of RAM, until Epic showed M$ Gears of War with and without the extra 256MB of RAM.

M$ decided to take a 1 billion loss right there to put the extra memory.

BlueBarad

I remember that, there is no disputing that the 512MB makes a world of difference. Or M$ sure as hell would not have dropped the billion down for taking Epic's advice. I am just under the impression that even though the PS3' memory is split into 256MB/256MB, through the unified architecture, it shares the load.

I have read how this creates a particular bottleneck of sorts, but what I am trying to understand is that if the split memory is unified through it's unified architecture, then how big can the hit really be if they went ahead with the design after what I am sure was painstaking research and development?

[QUOTE="Redfingers"][QUOTE="lordxymor"][QUOTE="Mortok"]

PC games span accross multiple CDs, not DVDs or Blueray, the gameplay is still enjoyable, Graphics is still beautiful and the story is still immersive. Your arguement has no bearing on this topic, the TC isn't attacking any platform but he is explaining the misconception of multi-disk games. He's explaining what PC gamers have known for awhile.

World of Warcraft, Company of Heroes, World in Conflict, Unreal Tournament 3 and Crysis all make Killzone 2's 2 gig level design .doc seem like trans fat -- neither required nor beneficial.

Datheron

But PC games are installed, when they are decompressed. If you compress a console game in the same levels of PC games, you're gonna waste mad ammounts of memory and cpu to decompress everything in real-time.

Here's where the argument gets interesting. Warhawk takes up 800 MB cache extremely compressed on the hard drive, both the retail and downloadable versions.

The developer said in an interview with IGN that this 800 megs stays 800 megs until the very moment you decide to demand the content, at which point it sits on main memory. Main memory meaning 256 MB XDR memory, I presume. How this works I do not know but the game works without a single hitch and it does indeed work.

It's a common misconception that compression takes huge amounts of memory + CPU power. It takes some, but the slowness you see in programs like Winzip are disk-bound, not CPU-bound. (i.e., it's slow because it's writing to disk) Furthermore, certain things are more compressable than others. A quick compression lesson:

Suppose your data is 11110000. If your compression scheme is 1 = 1111 and 0 = 0000, you can write "10" on your disk and be done.

Suppose your data is now 10101110. It's not as uniform, so you might do something like 1 = 10, 0 = 11 and you'd have to write "1101" on your disk to make it work.

With millions of bytes, there are algorithms that approximate the best schemes for the data, but you get the idea. In general, media is highly compressible (even lossless, and we can go really far w/ lossy compression), level information is also high compressible (think of points of a polygon, a lot of coordinates in a real game "line up"), but code isn't.

I know IO is the biggest bottleneck in decompressing, but depending on your compression algorithm, what your trying to decompress and how it can also take a lot of resources and processing, specially considering the very limited memory on consoles..

Ie: decompressing multi-layered high resolution textures, lossy compressed audio or the most classical example, decompressing advanced profile mpeg4 video.

http://www.psu.com/Killzone-2-needs-Blu-ray,-says-Guerilla--a0001149-p0.php

http://www.psu.com/2GB-levels-in-Killzone-2--a0001111-p0.php

The first link has the developers saying they would be hard pressed to fit the game on the Xbox 360....and that they need Blu Ray for games. The second says that each and every level would be 2 gigabytes according to the developers.

http://www.psu.com/Killzone-2-needs-Blu-ray,-says-Guerilla--a0001149-p0.php

http://www.psu.com/2GB-levels-in-Killzone-2--a0001111-p0.php

The first link has the developers saying they would be hard pressed to fit the game on the Xbox 360....and that they need Blu Ray for games. The second says that each and every level would be 2 gigabytes according to the developers.

Redfingers

Pardon me, I looked at the Gamespy source material and it sounds like a case of dumb PSU editing. Gamespy says the size of a single level is 2 gigabytes, and that's all.

[QUOTE="Datheron"]There are perfectly legitimate reasons for storing levels as a whole unto themselves on a disk. R:FoM showed that it's simply more efficient to read off of the BluRay continuously instead of repeatedly seeking + reading data, and in this case the size of the media compensates for its latent access.Velocitas8

I see, that sounds reasonable. Does it really make that much of a difference, though? I'm not terribly familiar with optical storage, but does sector seek-time really make up that much of a game's loading time? Either way I suppose any bit of benefit would be enough to justify its usage, since Blu-Ray has massive amounts of storage space to throw around anyways.

And wouldn't the HDD caching of game resources (since the HDD is standard on the PS3) help significantly cut down on seeking and loading times, too? Though obviously you have to move the resources from the disc to the cache in the first place in order to see any benefit.

Yea, seek times are terrible with optical disk drives because they cannot sustain the spin rates that internal hard drives spin at, due to of course the loose nature of being able to insert disks in the first place.

Also this:

Overall, CD-ROM drives have much poorer seek time performance than hard disks do. They use a much less efficient head actuator mechanism that causes them to take much more time to position to different tracks on the surface of the disk. This is probably because of the legacy of CD-ROM technology, which started out with audio CDs. When listening to an audio disk the only time a random seek is done is when changing music tracks, which is done very rarely.

And yes, HD caching is a strategy that's ben employed in PC games forever and on some PS3 games as well - I think even Ridge Racer 7 takes 5GB of hard drive space on the PS3 if you let it.

Excellent post, one of the best I've ever read in SW.

I certainly feel more edumacated :)

Hm, we happen to have similar-looking names, DethZero. Coincidence, or my alter ego? :-)

It is true that diskspace is mainly used for media such as videos, sounds, and textures. Same for code.

There are couple of things you have not touched and not everything is correct. For example, you stated about models taking up space, they dont. It's the textures you use on them that do. Also using very high resolution textures is pretty bad too.DethZero

Care to explain what you mean? Models take up space because you need to record, at the very least, the points in threespace which make up the model. It may be cheap compared to other things, but it's still space. I have no idea what you mean by high-res. textures being bad.

Just because you have 50gb of storage space, doesn't mean you can use them uncompressed everywhere. Remember, your video buffer might be limited to 128 to 256mb of video ram reserved depending on the system you develope. More true for the consoles than on PC. You DON'T want to go over that because now you'll be dealing with virtual memory reserved and another bottleneck to deal with which could sacrifice your performance. Also with higher resolution textures, it stresses the GPU more now because it has more data to compute and render. I always feel that a higher polygon count is worth more than a better texture because it will seem more fluid from an artistic point of view. However, one thing I notice alot about next gen games is that they dump all the high poly and textures on models and yet the environments look like something from 5 years ago.DethZero

Agreed, and you forgot to mention the time it takes to transfer large texture files from disk to VRAM. That said, current texturing technology - anisotropic filtering and bump mapping, for example - is advanced enough that significant rendering time is saved by processing the texture intelligently rather than build the geometry exactly. Stuff like cracks in a brick are great for textures; a game such as LittleBigPlanet really shows what high resolution textures can affect how sharp a game looks.

I remember when I worked on Transport Giant for the PC. Our maximum buffer for graphics was 90mb. We filled that up within couple of hours and it only contained about maybe 1/20th of the game. So we made pretty early screenshots using those. Of course we had to start compressing and lowering the quality to fit everything into that 90mb limit. Do note, this is for the PC and that time we were aiming for 128mb video card recomendations.DethZero

Hm...not familiar with that game, although I have very fond memories of Transport Tycoon by Microprose back in the old days.

Console dev is different than PC, though, because you know exactly how much RAM you have at your disposal, and you can start doing stuff like stream later portions of levels as the player is playing through the earlier ones because you can pick a memory replacement scheme that makes the most sense. The God of War II devs, for instance, explain in their developer commentary vids how they placed empty sections in between levels so they had time to stream the next area's data.

edit: also one thing I forgot to mention, when working on a PC, all the game components were installed on the harddrive so there was nothing needed to read from the CD except for a cd-check script right at the beggining when launching the game. For consoles, the speed and seek-time of the drive will make a HUGE difference in terms of loading the game information into memory. Having an active stream from a media source is also a bad programming decision so it's best to make sure that all your gama data can fit into ram and virtual memory.DethZero

Well, like I said above, the nice thing about working on consoles is that you know what kind of hardware you're dealing with so you can do some exotic things to make your game run smoothly even with the hardware limitations. Stuff like UT's 200MB maps being completely loaded into RAM at start just don't happen on consoles; the PS2 had a paltry 2MB of space to work with!

[QUOTE="Redfingers"]

Silence, Guerilla Games is speaking. Let's listen in:

"One level in Killzone 2 takes up 2 gigabytes."

thirstychainsaw

Wasn't the same things said about Lair and Heavenly Sword?

Exactly. Redfingers, a bigger game (more audio or whatever) doesn't make a BETTER game. Lair is proof of this. You can try and defend Blu-Ray all you want. But the fact of the matter is that compression techniques, plus good programmers, can make a DVD-9 stretch very far. Take your head out of Sony's ass and wake up.

Good post. Does this mean then that either Lost Oddessy has a lot of cut scenes, has a lot of areas/different building/different trees, or is a really big game?

Let me preface by saying that Guerilla followed the "2 gigabyte" comment by saying that the game will absolutely not fit on anything less than a Blu Ray disc. The only legitimate argument against this is that, essentially, he's either lying or paid by Sony to say that...

I'm just not certain he's bloating the disc up with arbitrary information. He said, "yes, the level is 2 gigabytes, and, it follows that we cannot fit the game on a DVD-9." That assumes there's plenty where that came from, be it textures, layout, etcetera.

I really cannot force myself to believe even for a moment that that figure is erroneous.Redfingers

Sure. I don't believe he's lying either, but I do think what's happening is that he said to disparate things and you pieced them together, whether it was his intention or not.

Let's do some back-of-the-envelope calculations. Suppose that level lasts 30 min., and KZ2 will last 10 hours, not reasonable numbers. Then we're talking 20 of these 2-GB-on-average levels being put in a game, which, if we go by what you're saying, would be 40GB alone! KZ2 looks impressive, sure, but I highly doubt it contains so much data that it blows every other shooter in existence out of the water in storage requirements. Now, they may have 10GB, 12GB of level data - that to me is much more reasonable and the statement "would not fit on a DVD9" would still hold true.

Now that the thread is stickied, no one is going to bother reading :P

Now that the thread is stickied, no one is going to bother reading :P

Eponique

[QUOTE="Eponique"]

Now that the thread is stickied, no one is going to bother reading :P

Greyhound222

That's the plan anyway.

[QUOTE="Greyhound222"][QUOTE="Eponique"]

Now that the thread is stickied, no one is going to bother reading :P

cakeorrdeath

That's the plan anyway.

Needs "Disc Usage" to be in caps. (please note correct spelling of DISC)

[QUOTE="Greyhound222"][QUOTE="Eponique"]

Now that the thread is stickied, no one is going to bother reading :P

cakeorrdeath

That's the plan anyway.

Kudos cake, nice find, great thread

Good post. Does this mean then that either Lost Oddessy has a lot of cut scenes, has a lot of areas/different building/different trees, or is a really big game?

SgtWhiskeyjack

all of the above