Unison is a file-synchronization tool for Unix and Windows. It allows two copies of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other. Unison can deal with [...]

Tags: data synchronize, load balance, load balancing, mirroring, rsync

• 0 Comments

MogileFS is our open source distributed filesystem. http://danga.com/mogilefs/ Some of the advantages are No Single point Failure Automatic file replication Better than RAID Shared-Nothing — MogileFS doesn’t depend on a pricey SAN with shared disks. Every machine maintains its own local disks. No RAID required — Local disks on MogileFS storage nodes can be in [...]

Tags: distributed filesystem, MogileFS, raid, scaling

• 0 Comments

WordPress.com use Amazon S3 for storing images, they do not serve images directly from Amazon, they use Varnish reverse proxy at front end, so not all images need to be served from S3, Varnish cache popular images and serve directly from its cache. Many big web sites use varnish or squid at front end as [...]

Tags: reverse proxy, scalability, squid, varnish

• 1 Comment

Many large web sites use reverse proxy servers in front end. For example flicker use squid in reverse proxy mode, wordpress use varnish in front end. Varnish was written from the ground up to be a high performance caching reverse proxy. Squid is a forward proxy that can be configured as a reverse proxy. Besides [...]

Tags: cache, reverse proxy, varnish

• 0 Comments