b) Generation of the Hi-C ligation junction sequence by successive digestion (with HindIII in this example), fill in and blunt-ended ligation steps. The red and blue rectangles represent cross-linked restriction fragments while the yellow marker shows the position of biotin incorporation. Since these two fragments were positioned close to each other during fixation, by analysing the composition of a population of di-tags generated by a Hi-C experiment it is possible to infer genomic three-dimensional organisation.Ī) Diagram summarising the Hi-C experimental protocol. The resulting molecule, termed a di-tag, should comprise two different DNA fragments separated by a modified restriction site. Following sonication the sheared ligated DNA fragments are enriched by streptavidin pull-down of the biotin residues, and then are ligated between sequencing adapters. Fragments in close spatial proximity are ligated together generating a novel "modified restriction site" sequence (see Figure 1b). Overhanging single-stranded DNA at the ends of restriction fragments are then filled in with the concomitant incorporation of biotin. The method (summarised in Figure 1a) involves fixing chromatin to preserve genomic organisation, followed by restriction enzyme digestion of the DNA. But deduping could cut your home or business storage requirements by a factor of 10 I suspect that smaller businesses with fewer users won't achieve ratios up to 20:1 as larger ones do.Hi-C is a ligation-based proximity assay utilising the power of massively parallel sequencing to identify three-dimensional genomic interactions 1. That makes deduping, rather than buying more storage, economically viable.įor the rest of us, a 1TB disk costs well under a hundred quid and looks like the way to go, for now. The problem is that enterprise-level data is hugely expensive because it meeds to be surrounded by management software, by redundant components and by other subsystems that ensure that not a single bit is lost. There's a lot of voices calling for a Linux-based deduping file system - there's even one, lessfs, under development but it's still in beta, and I wouldn't trust my backups to a beta-level file system - would you?īut the signs are that Linux will sprout at least one such add-on and may eventually include such functionality in the kernel. What about the rest of us? Smaller businesses will have to wait. If your network can stand it, then this is the route you're more likely to take. You pay for source-level deduping with network traffic but you keep the CPU firepower under control in the datacentre. It means you can't do it on every client: not all of them have the horsepower, and a 10-20 percent CPU hike during backups can adversely affect the end-user experience.Īs a result, many users deduplicate at the target, using dedicated CPU-heavy boxes, often acting as virtual tape libraries - storage that looks to the backup software like a tape but in fact consists of disks. This, if you move the process of deduplication back towards the source, becomes increasingly important, as the process is highly processor-intensive. Deduping at source saves the network from carrying lots of duplicate traffic but loads up the clients. It was developed as the idea that all storage problems can be sorted with the application of yet more storage started to become economically unviable.Īrguments rage over whether deduplication should take place at the source or the target. So enterprises use deduplication mostly in backup devices, rather than live, tier one storage in production environments. The problem is that corporate data is growing at a rate that outstrips the ability and willingness of enterprises to pay for it. Advocates reckon that compression levels of around 10:1 to 20:1 can be routinely achieved. Yet signs are it won't be too long before it becomes commonplace.Īs I mentioned in an earlier blog, deduplication is the process of storing only one copy of a lump of data - be it a file or a block - and when a duplicate of that data needs to be stored, creating a tiny pointer to it instead. I found myself thinking about storage deduplication recently, while wondering why only large enterprises can afford this technology.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |