Support SQL metadata engine, JuiceFS v0.13 released

After 1 month and involving more than 80 changes, JuiceFS v0.13 is finally here!🎉

Let's take a look at the changes in the new version:

Support SQL metadata engine

In addition to the regular bug fixes, the biggest change brought about by this upgrade is to start to support the SQL database as a metadata engine. The following are the metadata engines currently supported by JuiceFS:

• Redis
• MySQL
• MariaDB
• TiDB
• SQLite

Relational databases have slightly lower performance than Redis, but relational databases can provide better data persistence and scalability, so they are more suitable for scenarios that are more sensitive to the total amount and security of data, such as file backup.

In terms of functionality, MySQL, MariaDB, and TiDB can all provide functional features that are almost the same as Redis, such as multi-client shared storage and strong data consistency. Moreover, MySQL and its protocol-compatible database products have a huge user base and a complete supporting ecosystem, which can be easy to use and management.

It is worth mentioning that the support for SQLite database allows JuiceFS to further expand the applicable application scenarios. As the most popular single-file database in the world, SQLite is easy to use and requires no configuration before use. It is very suitable for single-device or personal small-scale use. For example, you can use the combination of "SQLite + Object Storage" to quickly create a JuiceFS storage, mount it to your local computer and use it as a private network disk.

New sub-commands

JuiceFS has also added a series of useful sub-commands:

• profile
• status
• warmup

profile

This command is designed to monitor the working status of JuiceFS. Basic usage:

$ juicefs profile MOUNTPOINT/LOGFILE

Assuming that JuiceFS storage is mounted in the /jfs folder, you can use the following command to monitor the operations on this directory in real time:

$ juicefs profile /jfs

You can also specify the .accesslog log file to replay the historical operations:

$ cat /jfs/.accesslog > /tmp/jfs-oplog
# later
$ juicefs profile /tmp/jfs-oplog

status

This command can be used to view the detailed information of a JuiceFS storage, such as name, metadata address, storage type, bucket, compression algorithm, etc. Basic usage:

$ juicefs status [command options] Metadata-database-URL

For example:

$ juicefs status mysql://user:passwd@(localhost:3306)/juicefs

2021/05/20 14:50:38.500683 juicefs[3259538] <INFO>: Meta address: mysql://user@(localhost:3306)/juicefs
[xorm] [info]  2021/05/20 14:50:38.503441 PING DATABASE mysql
2021/05/20 14:50:38.504942 juicefs[3259538] <WARNING>: The latency to database is too high: 1.504649ms
{
  "Name": "juice-test",
  "UUID": "1324d1bf-98f0-4e97-a4c1-60bd7c04debf",
  "Storage": "minio",
  "Bucket": "https://192.168.1.8/mybucket",
  "AccessKey": "minioadmin",
  "BlockSize": 4096,
  "Compression": "none",
  "Shards": 0,
  "Partitions": 0
}

warmup

This command can manually specify to build a local cache for a specific directory or file, so that it can speed up the access of data in JuiceFS storage.

$ juicefs warmup -h
NAME:
   juicefs warmup - build cache for target directories/files

USAGE:
   juicefs warmup [command options] [PATH ...]

OPTIONS:
   --file value, -f value     file containing a list of paths
   --threads value, -p value  number of concurrent workers (default: 50)
   --background, -b           run in background (default: false)
   --help, -h                 show help (default: false)

Other changes

In addition to the previously, JuiceFS v0.13 also brings the following new features:

• Java SDK for Windows
• Supports shards, which can store file blocks in multiple buckets.
• Add a /.stats virtual file to count the storage infomation.
• Add a minimal compilation option make juicefs.lite, so that the client only supports local and S3 storage, and the size of the client after compilation can be reduced by 2/3.

For more changes, please check the GitHub release page.