The entire concept of this blog came after my participation in a thread at LinkedIn where someone was asking for a 500-600TB Storage for a CCTV project.
I took an active part in this thread to some extent. I stopped, as I felt it is useless.
This brings me to discuss some common myths / believes / wrong concepts storage people commonly make regarding customers looking for storage, and storage for CCTV usage specifically.
I would like to try and blow the air out of some of these balloons.
I will randomly go through some wrong concepts & will try enlighten as to why these concepts are wrong.
#1 My storage can fit all your needs -
This is classic common mistake by storage people (whether on purpose or not), that is industry wide. A customer asks for a storage, the vendor guy, jumps the gun “MINE”, or offer some storage he thinks is *good* in his perception. Sometimes, this unprofessional attitude is not even followed even by discovery process of what the customer actually WANTS to achieve.
Not all storage systems made alike, and they serve different needs for different type of customers, with different set of requirements.
Capacities, density, functionality, scalability, redundancy, power, heat, weight, data protection, price, IOps, throughput, drive support, latency, yada yada yada…. Not all made alike.
#2: “Video ? That is sequential data, any storage can handle that” -
To some extent. Yes, video stream by nature is a sequence of frames or compressed objects that comes sequentially.
However, many people tend to forget that there is a small layer between the video stream & the storage called the *Application*. In our case, the NVR( Network-Video-Recorder).
There are hundreds of NVR applications out there. Just some known names : MileStone, DVTEL, NICE, Verint, MarchNetworks, IndigoVision, and more….
Each Application is written differently, and handles/manages storage differently.
Each application manages its data in different ways, some work sequntial, some work random in database.. Some use 64k blocks, while other do random 4k..
There is no one rule to how NVR software (Sadly) apply data to storage device, and therefore thinking that video is always applied to storage in a sequential manner is wrong at its basis.
#3 SAN vs NAS
Takes us right back to #2. In the thread mention, many storage folks jump the gun and offer various SAN or NAS solution, and argue about which is better / more suitable.
None of them bothered to ask what is the NVR application being used ? Does it support NAS access ? Does it support a file system at all ?
Some Vendors support SAN & NAS, some write to a file system, and some write to raw devices, some are POSIX compliance & some are not..
Again, there is no one rule & it is all application and project driven.
#4 Video Data protection should be done @ storage level replication/synchronization -
Storage people from the IT world, are used to the fact that data protection is done on the storage level, which many of the IT branded storage products provide replication mechanisms.
Is it true that this is the best method to protect video data to remote site ? Not always..
Disaster recovery & business continuity needs to be looked at from a complete project architecture level. Not always data protection on Storage is the best approach.
On the contrary, on CCTV, it makes the LEAST sense to replicate video data on storage level. Why :
a) Many times it is pointless to replicate continuous video data. It makes sense to replicate events.
b) It is much more simple to use multi – unicast or multicast to split a video stream to two sites, instead of overloading the storage device.
c) Data is constantly changing, and many use containers than are re-written, and therefore, the data is over-written constantly, replication may be irellevant or never catch up. (as usually these are large capacities)
d) Video files sometimes require their meta-data that needs to be replicated consistantly with the data itself. Sometimes it is not possible. Replicating video data without the ability to recover it is meaningless…
#5 iSCSI as a market leading protocol
iSCSI is , for sure, a strong player in the CCTV Market. You can see it all around in various products, especially in ones that position themselves as dedicated storage device for CCTV such as Pivot3, Intransa, DNF Security & more.
Why is iSCSI so popular. My take on this, is that iSCSI was the easiest extension to the Direct Attached Storage (or internal drives) of the 3-4yrs ago CCTV market.
The pre-Mega-Pixel era didn’t require so much storage space, so internal disks where fine. Suddenly, these same application need to handle massive amounts of storage per NVR, due to retention periods & resultion. However, the applications are written and suitable to use internal drives.
iSCSI comes as a very natural & easy to deploy extent to internal drives, making them external but with similarity to what the integrators in this market already know.
This is a Video based market, not IT / infrastructure based market, so it makes much sense for these guys to choose the next most simple storage solution, that works similar to DAS.
Is it bad ? Not necessarily. iSCSI can be good cost-effective solution to many projects, especially small to medium scales.
However, when a NVR needs to see large quantities of storage, and when a projects scales out, that is where iSCSI limitations start to pop (and I will get into details on that on another post). Congestion & lun spaggeti mgmt, make a scalable project a night mare… With some many NVR application still in the Windows 32bit space, this becomes even more annoying with the lun size limitations they imply.
There are many more & I may add more later on, as my brain refreshes.
Feel free to throw in questions / other myth you want clarified / comments.
Ran
