There have been many questions about how to set up your disks.

Where do I put my media cache files, where the page file, and what about my preview files?

All these allocations can be set in PR, so I made this overview to help you find some settings that may be beneficial. It is not a law to do it like this, it is a generic approach that would suit many users, but depending on source material, workflow and backup possibilities, it is not unthinkable you need to deviate from this approach in your individual case.

The reasoning behind this overview is that you want to distibute disk access across as many disks as possible and get the best performance.

Look for yourself:

I hope this helps to remove doubts you may have had about your setup or to find a setup that improves performance.

Adobe has on its website the Adobe Premiere Pro CS5: System Requirements

Unfortunately, this overview is severely lacking in realism and a lot of people were disappointed when their system met these minimum requirements, but still would not do what they expected, or at least not without jerkiness, hiccups and similar issues.

The problem with these requirements as stated is that they are really bare minimum requirements to install the software, but unfortunately it does not tell how well a certain system will perform with the myriad of codecs used and the different needs and expectations people may have about their editing rig. Since this issue is nearly a year old now and nothing tangible has changed, I decided to write this article to help people understand what is realistic to expect, what influences the hardware choices in order to use CS5 to full satisfaction.

CAVEAT: This is my personal opinion, in no way authorized or endorsed by Adobe, who have not seen anything I write here till the moment it has been published here.

Context:

The nature of one’s editing projects can have a major impact on the hardware required to run projects effectively. Long form documentaries, delivered on BRD demand different hardware and priorities in hardware setup then music clips with lots of multicam work and color effects delivered to the web, or wedding video’s delivered on DVD. And unfortunately, there is no simple rule saying that if you edit X, you need Y hardware.

The second thing that has a major impact is the source material, the codec used. Back in the old days things were very simple, you had DV material from a tape based camera and that was it. Nowadays, things have grown much more complex. The number of codecs used in source material has grown enormously, the number of formats and frame rates has grown in a similar fashion. We used to have 480, now we have added 720, 1080, 2K, 3K, 4K plus various DSLR and other formats, we had interlaced, now we have progressive as well, we used to have 25 or 29.97 FPS, now we have 24, 25, 29.97, 60 and even more frame rates. We had DV, now we have MPEG2, HDV, XDCAM, P2, AVCHD, RED, Cineform, Matrox, and numerous other codecs.

This makes it all the more confusing for people to know what they need when they start out with video editing to run CS5 successfully.

The codec issue:

Some codecs are easy to handle for a computer, others are difficult to handle. It is generally known that DV material is very easy to handle and AVCHD is pretty tough to handle. The general rule is that the more compressed the material is, the harder it is for the computer to edit this. GOP (Group of Pictures) structure is an extra burden. The higher the resolution, the harder it is on the computer.

To simplify matters one could differentiate codecs and source material in three categories, based on their properties:

Of course boundaries when using three categories are not always very clear, but the tendency is rather clear. Easy codecs are in the upper left corner, difficult codecs are in the lower right corner. For that reason we have identified three categories, Easy, Intermediate and Difficult.

This is somewhat similar to the color coding PR uses with none, yellow and red in the time line. It is not complete in the overview of common codecs, but is intended to show what the impact can be of different codecs and the hardware requirements.

I realize this is a limited overview and generic in nature. It will not answer all questions, but may be helpful to avoid disappointments. It also has a number of limitations. For instance, uncompressed MS AVI SD material. No GOP structure, no compression, low resolution, that should be at the top left corner, right? Wrong. Uncompressed does not burden the CPU, but is a definite burden on the memory and disk I/O system.

What does that mean in terms of requirements?

System requirements:

If the codecs you generally use are slightly more than just plain “easy”, look at the improvemnts in the next column and repeat for the next column if applicable.

I hope this can benefit people to understand there is more to choosing a system than just following the Adobe site.

Additions or suggestions for improvements are welcome.

Who are they and what’s your favorite ??

More experience than our name implies

Keep climbing

Fly the friendly skies

Work hard, fly right

Fly smart

The joy of flying

No ordinary airline

Up up and away

I have a new BlackMagic HyperDeck Shuttle, the output file format is a MOV file that is uncompressed 1920 x 1080 10-bit 4:2:2.  When I put this output clip into Premiere here is what the properties of the clip show.

What bothers me is the “Average Data Rate” of 90.8 MB/second.  Is this what I should be seeing?  I was under the impression that uncompressed HD 10-bit 4:2:2 required much higher.data rates something like 180 MB/second and thereby necessitating RAID arrays for Premiere.

Does anyone have uncompressed HD data to compare to my results?  The data is complete as I did also record the regular AVCHD data and have both in a timeline

i JUST BUILT A 6 CORE 970 @ 3.2 GHZ, ASUSTeK Computer INC. SABERTOOTH X58 Rev 1.xx

24 GIGS OF 1600 RAM, 2 SATA RAID 0, 1 10,000 RPM SYS DRIVE, 1 EXTRA SATA DRIVE. CS5.5 editing avchd.

NVIDIA GeForce GTX 570

programs on system drive

Scratch/page/renders on a different single sata

Video/Audio on Raid 0  2, 7200rpm sata drives

i NEVER OVERCLOCKED ANYTHING. Are there specific changes I could make, values i could change that would increase the speed of my machine without sacrificing stability. in other words, a formula of sort that others have followed with this particular MB/CPU?

Thanks:-)

Steve from NY

Here, on the forum the SSD is no go, no benefits to PREpro or AE, and I get why.

Now via Twitter I get this Adobe tweet about the benefits of SSD to us..

http://adobe.ly/qEIvrZ http://fb.me/1cqcfMA5o

What your opinion?

I think, it’s the same HP, Intel, Adobe deal like always.. If right, i feel sorry for the customers who aren’t diving in this great forum for a few hours.

Why? Mac OS X

I switched because of Adobes claim of using mutliprocessing on new Macs .. and posts that said FCP does not.

Tonight I found out the main reason I bought a 12 Core Mac Pro and the Master Suite ( Stabilization ) ..  the effect / rendering only uses ONE Half of ONE core .. as seen in the Activity Monitor .. and it is impossible to use any of the other cores in a 12 core multiprocessor Mac Pro.

Of course there is many other benefits of the 12 Cores .. but stabilization is not one of them.

Unfortunately it is my Number One time consuming ‘effect’.

Think Positive.

Right.

People often ask: Should I raid my disks?

The question is simple, unfortunately the answer is not. So here I’m going to give you another guide to help you decide when a raid array is advantageous and how to go about it. Notice that this guide also applies to SSD’s, with the expection of the parts about mechanical failure.

What is a RAID?

RAID is the acronym for “Redundant Array of Inexpensive Disks”. The concept originated at the University of Berkely in 1987 and was intended to create large storage capacity with smaller disks without the need for very expensive and reliable disks, that were very expensive at that time, often a tenfold of smaller disks. Today prices of hard disks have fallen so much that it often is more attractive to buy a single 1 TB disk than two 500 GB disks. That is the reason that today RAID is often described as “Redundant Array of Independent Disks”.

The idea behind RAID is to have a number of disks co-operate in such a way that it looks like one big disk. Note that ‘Spanning’ is not in any way comparable to RAID, it is just a way, like inverse partitioning, to extend the base partition to use multiple disks, without changing the method of reading and writing to that extended partition.

Why use a RAID?

Now with these lower disks prices today, why would a video editor consider a raid array? There are two reasons:

1. Redundancy (or security)

2. Performance

Notice that it can be a combination of both reasons, it is not an ‘either/or’ reason.

Does a video editor need RAID?

No, if the above two reasons, redundancy and performance are not relevant. Yes if either or both reasons are relevant.

Re 1. Redundancy

Every mechanical disk will eventually fail, sometimes on the first day of use, sometimes only after several years of usage. When that happens, all data on that disk are lost and the only solution is to get a new disk and recreate the data from a backup (if you have one) or through tedious and time-consuming work. If that does not bother you and you can spare the time to recreate the data that were lost, then redundancy is not an issue for you. Keep in mind that disk failures often occur at inconvenient moments, on a weekend when the shops are closed and you can’t get a replacement disk, or when you have a tight deadline.

Re 2. Performance

Opponents of RAID will often say that any modern disk is fast enough for video editing and they are right, but only to a certain extent. As fill rates of disks go up, performance goes down, sometimes by 50%. As the number of disk activities on the disk go up , like accessing (reading or writing) pagefile, media cache, previews, media, project file, output file, performance goes down the drain. The more tracks you have in your project, the more strain is put on your disk. 10 tracks require 10 times the bandwidth of a single track. The more applications you have open, the more your pagefile is used. This is especially apparent on systems with limited memory.

The following chart shows how fill rates on a single disk will impact performance:

Remember that I said previously the idea behind RAID is to have a number of disks co-operate in such a way that it looks like one big disk. That means a RAID will not fill up as fast as a single disk and not experience the same performance degradation.

Now that we have established the reasons why people may consider RAID, let’s have a look at some of the basics.

Single or Multiple?

There are three methods to configure a RAID array: mirroring, striping and parity check. These are called levels and levels are subdivided in single or multiple levels, depending on the method used. A single level RAID0 is striping only and a multiple level RAID15 is a combination of mirroring (1) and parity check (5). Multiple levels are designated by combining two single levels, like a multiple RAID10, which is a combination of single level RAID0 with a single level RAID1.

Hardware or Software?

The difference is quite simple: hardware RAID controllers have their own processor and usually their own cache. Software RAID controllers use the CPU and the RAM on the motherboard. Hardware controllers are faster but also more expensive. For RAID levels without parity check like Raid0, Raid1 and Raid10 software controllers are quite good with a fast PC.

The common Promise and Highpoint cards are all software controllers that (mis)use the CPU and RAM memory. Real hardware RAID controllers all use their own IOP (I/O Processor) and cache (ever wondered why these hardware controllers are expensive?).

There are two kinds of software RAID’s. One is controlled by the BIOS/drivers (like Promise/Highpoint) and the other is solely OS dependent. The first kind can be booted from, the second one can only be accessed after the OS has started. In performance terms they do not differ significantly.

For the technically inclined: Cluster size, Block size and Chunk size

In short: Cluster size applies to the partition and Block or Stripe size applies to the array.

With a cluster size of 4 KB, data are distributed across the partition in 4 KB parts. Suppose you have a 10 KB file, three full clusters will be occupied: 4 KB – 4 KB – 2 KB. The remaining 2 KB is called slackspace and can not be used by other files. With a block size (stripe) of 64 KB, data are distributed across the array disks in 64 KB parts. Suppose you have a 200 KB file, the first part of 64 KB is located on disk A, the second 64 KB is located on disk B, the third 64 KB is located on disk C and the remaining 8 KB on disk D. Here there is no slackspace, because the block size is subdivided into clusters. When working with audio/video material a large block size is faster than smaller block size. Working with smaller files a smaller block size is preferred.

Sometimes you have an option to set ‘Chunk size’, depending on the controller. It is the minimal size of a data request from the controller to a disk in the array and only useful when striping is used. Suppose you have a block size of 16 KB and you want to read a 1 MB file. The controller needs to read 64 times a block of 16 KB. With a chunk size of 32 KB the first two blocks will be read from the first disk, the next two blocks from the next disk, and so on. If the chunk size is 128 KB. the first 8 blocks will be read from the first disk, the next 8 block from the second disk, etcetera. Smaller chunks are advisable with smaller filer, larger chunks are better for larger (audio/video) files.

For a full explanation of various RAID levels, look here: http://www.acnc.com/04_01_00/html

Not using RAID entails the risk of a drive failing and losing all data. The same applies to using RAID0 (or better said AID0), only multiplied by the number of disks in the array.

RAID1 or 10 overcomes that risk by offering a mirror, an instant backup in case of failure at high cost.

RAID3, 5 or 6 offers protection for disk failure by reconstructing the lost data in the background (1 disk for RAID3 & 5, 2 disks for RAID6) while continuing your work. This is even enhanced by the use of hot-spares (a double assurance).

PERFORMANCE

RAID0 offers the best performance increase over a single disk, followed by RAID3, then RAID5 amd finally RAID6. RAID1 does not offer any performance increase.

Hardware RAID controllers offer the best performance and the best options (like adjustable block/stripe size and hot-spares), but they are costly.

SUMMARY

If you only have 3 or 4 disks in total, forget about RAID. Set them up as individual disks, or the better alternative, get more disks for better redundancy and better performance. What does it cost today to buy an extra disk when compared to the downtime you have when a single disk fails?

If you have room for at least 4 or more disks, apart from the OS disk, consider a RAID3 if you have an Areca controller, otherwise consider a RAID5.

If you have even more disks, consider a multilevel array by striping a parity check array to form a RAID30, 50 or 60.

If you can afford the investment get an Areca controller with battery backup module (BBM) and 2 GB of cache. Avoid as much as possible the use of software raids, especially under Windows if you can.

RAID, if properly configured will give you added redundancy (or security) to protect you from disk failure while you can continue working and will give you increased performance.

Look carefully at this chart to see what a properly configured RAID can do to performance and compare it to the earlier single disk chart to see the performance difference, while taking into consideration that you can have one disks (in each array) fail at the same time without data loss:

Hope this helps in deciding whether RAID is worthwhile for you.

WARNING: If you have a power outage without a UPS, all bets are off.

A power outage can destroy the contents of all your disks if you don’t have a proper UPS. A BBM may not be sufficient to help in that case.

I just wrote up a very long post and I seem to have trouble posting it — “Cannot post now” error, even though I can post in the testing forum. This is a short post to see if it works…

Just when you thought you’d take a nice Summer auto trip – scary snakes attack!!!!!

http://www.theblaze.com/stories/ooooh-my-god-snake-on-windshield-trigg ers-highway-freak-out/

Hunt

PS – this might cut into the sales of convertibles…