我的书签
添加书签
移除书签Comparing Storage Options
The Filer Store persists all file metadata and directory information.
| Filer Store Name | Lookup | number of entries in a folder | Scalability | Renaming | TTL | Note |
|---|---|---|---|---|---|---|
| memory | O(1) | limited by memory | Local, Fast | for testing only, no persistent storage | ||
| leveldb | O(logN) | unlimited | Local, Very Fast | Default, fairly scalable | ||
| leveldb2 | O(logN) | unlimited | Local, Very Fast, faster than leveldb | Similar to leveldb, part of the lookup key is 128bit MD5 instead of the long full file path | ||
| Redis | O(1) | limited | Local or Distributed, Fastest | Yes | one directory's sub file names are stored in one key~value entry | |
| Cassandra | O(logN) | unlimited | Local or Distributed, Very Fast | Yes | ||
| MySql | O(logN) | unlimited | Local or Distributed, Fast | Atomic | Easy to manage | |
| Postgres | O(logN) | unlimited | Local or Distributed, Fast | Atomic | Easy to manage | |
| MemSql | O(logN) | unlimited | Distributed, Fast | Atomic | Scalable | |
| TiDB | O(logN) | unlimited | Distributed, Fast | Atomic | Scalable | |
| CockroachDB | O(logN) | unlimited | Distributed, Fast | Atomic | Scalable | |
| Etcd | O(logN) | ~10GB | Distributed, 10,000 writes/sec | No SPOF. High Availability. | ||
| TiKV | O(logN) | unlimited | Local or Distributed, Fast | Atomic | Easy to manage |
Switching between different Stores
It is easy to switch between different filer stores.
For example:
- # first save current filer meta data
- $ weed shell
- > fs.cd http://filerHost:filerPort/
- > fs.meta.save
- ...
- total 65 directories, 292 files
- meta data for http://localhost:8888/ is saved to localhost-8888-20190417-005421.meta
- > exit
- # now switch to a new filer, and load the previously saved metadata
- $ weed shell
- > fs.meta.load localhost-8888-20190417-005421.meta
- ...
- total 65 directories, 292 files
- localhost-8888-20190417-005421.meta is loaded to http://localhost:8888/
Extending Storage Options
For any new storage option, please implement the FilerStore interface. It should be fairly straight forward to implement. Welcome to contribute back.
