Home Download Docs Code Community

Camlistore Overview

Camlistore is your personal storage system for life.

Summary

The project began because I wanted to...

Most of this works as of the 0.1 release, and the rest and more is in progress.

Longer Answer

Throughout our life, we all continue to generate content, whether that's writing documents, taking photos, writing comments online, liking our friends' posts on social networks, etc. Our content is typically spread between a mix of different companies' servers ("The Cloud") and your own hardware (laptops, phones, etc). All of these things are prone to failure: companies go out of business, change ownership, or kill products. Personal harddrives fail, laptops and phones are dropped.

It would be nice if we were a bit more in control. At least, it would be nice if we had a reliable backup of all our content. Once we have all our content, it's then nice to search it, view it, and directly serve it or share it out to others (public or with select ACLs), regardless of the original host's policies.

Camlistore is a system to do all that.

While Camlistore can store files like a traditional filesystem (think: "directories", "files", "filenames"), its specialized in storing higher-level objects, which can represent anything..

In addition to an implementation, Camlistore is also a schema for how to represent many types of content. Much JSON is used.

Because every type of content in Camlistore is represented using content-addressable blobs (even metadata), it's impossible to "overwrite" things. It also means it's easy for Camlistore to sync in any direction between your devices and Camlistore storage servers, without versioning or conflict resolution issues.

Camlistore can represent both immutable information (like snapshots of filesystem trees), but can also represent mutable information. Mutable information is represented by storing immutable, timestamped, GPG-signed blobs representing a mutation request. The current state of an object is just the application of all mutation blobs up until that point in time. Thus all history is recorded and you can look at an object as it existed at any point in time, just by ignoring mutations after a certain point.

Despite using parts of the OpenPGP spec, users don't need to use the GnuPG tools or go to key signing events or anything dorky like that.

You are in control of your Camlistore server(s), whether you run your own copy or use a hosted version. In the latter case, you're at least logically in control, analagous to how you're in charge of your email (and it's your private repository of all your email), even if a big company runs your email for you. Of course, you can also store all your email in Camlistore too, but Gmail's interface and search is much better.

Responsible (or paranoid) users would set up their Camlistore servers to cross-replicate and mirror between different big companies' cloud platforms if they're not able to run their own servers between different geographical areas. (e.g. cross-replicating between different big disks stored within a family)

A Camlistore server comprises several parts, all of which are optional and can be turn on or off per-instance:

Enough words for now. See the docs and code for more.

Last updated 2013-06-12

Website layout inspired by git and memcached,
design done by up all day creative solutions.
Content by the authors.