JuiceFS S3 Gateway

JuiceFS splits and upload files to the underlying object storage, applications often use the exposed POSIX API. But if you ever need to use S3-compatible API to access JuiceFS files, S3 Gateway comes in handy, its architecture:

S3 Gateway is usually used to:

• Expose S3 API for JuiceFS file system, so that applications may access JuiceFS via S3 SDK
• Use tools like s3cmd, AWS CLI and MinIO Client to access and modify files stored in JuiceFS
• When transferring data across regions, use S3 Gateway as an unified data export endpoint, this eliminates metadata latency and improve performance. See Sync across regions using S3 Gateway

Deploy S3 Gateway

Same as the mount command, gateway reads the local JuiceFS Client configuration file (defaults to ~/.juicefs/$VOL_NAME.conf), if this config doesn't yet exist, you need to authenticate against the Web Console using the auth command, this command will fetch the configuration file.

juicefs auth $VOL_NAME --token=xxx --access-key=xxx --secret-key=xxx

S3 gateway is implemented using the open source MinIO code, so you'll have to provide some MinIO related variables:

export MINIO_ACCESS_KEY="admin"
export MINIO_SECRET_KEY="password"

Forgetting these variables will result in errors like MINIO_ROOT_USER should be specified as an environment variable with at least 3 characters.

With the credential variables in place, run the gateway command to launch gateway:

juicefs gateway myjfs 127.0.0.1:8888

Expect the following output:

2023/03/21 20:15:49.945403 juicefs[97188] <INFO>: connected to 47.103.20.252:9308 [client.go:874]
2023/03/21 20:15:49.965411 juicefs[97188] <INFO>: Cache: /Users/herald/.juicefs/cache/jfs8 capacity: 102400 MB [disk_cache.go:747]
Endpoint: http://127.0.0.1:8888

Browser Access:
   http://127.0.0.1:8888

Object API (Amazon S3 compatible):
   Go:         https://docs.min.io/docs/golang-client-quickstart-guide
   Java:       https://docs.min.io/docs/java-client-quickstart-guide
   Python:     https://docs.min.io/docs/python-client-quickstart-guide
   JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
   .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide

Assuming your server's public IP being 111.2.3.4 and you want to open up access to the gateway over the internet and bind to port 9000, you can adjust the startup command like this:

juicefs gateway myjfs 111.2.3.4:9000

Once S3 Gateway is launched, you can directly access the URL you set in your browser, i.e., http://127.0.0.1:8888 in the example above. Open the file explorer for web, and enter the corresponding Access Key and Secret Key to login.

In addition, clients like AWS CLI and MinIO Client and S3 SDKs for various programming languages are also available for accessing S3 Gateway.

Enable virtual-hosted-style

Since S3 Gateway is developed upon opensource MinIO code, the MINIO_DOMAIN variable is supported, you can use this to enable virtual-hosted-style:

export MINIO_DOMAIN=mydomain.com
juicefs gateway myjfs 111.2.3.4:9000

If Helm chart (read below section) is used to deploy S3 Gateway, specify this variable in envs for the same effect.

Deploy S3 Gateway in Kubernetes

Installation requires Helm 3.1.0 and above, refer to the Helm Installation Guide.

helm repo add juicefs https://juicedata.github.io/charts/
helm repo update

Our Helm chart simultaneously support JuiceFS Community and Enterprise edition, distinguished by populating different fields in the values file. Take enterprise edition for an example, you can create a separate values file to overwrite these important fields:

values-mycluster.yaml

# Our default values.yaml uses community edition, change to enterprise edition
image:
  repository: juicedata/mount
  tag: "ee-5.0.21-f900c6e"

secret:
  name: "myjfs"
  # If the token field is populated, installation will be treated as enterprise edition
  token: "xxx"
  accessKey: "xxx"
  secretKey: "xxx"

tip

Don't forget to include the values-mycluster.yaml into your Git project (or using other source code management systems), so that all changes on the values file can be traced and rolled back.

Once credentials are configured, run the following command to deploy:

# Below command can be used both to carry out the initial installation, and future upgrades
helm upgrade --install -f values-mycluster.yaml s3-gateway juicefs/juicefs-s3-gateway

After installation, follow the output instructions to get the Kubernetes Service address, and verify if it's working.

S3 Gateway and symbolic link

There's no such thing as a symbolic in object storage, nevertheless, symbolic link is supported in JuiceFS. So if your file system contains any, pay attention when using S3 Gateway.

• All symbolic links (relative or absolute) should only target files within JuiceFS (file, not directory). If link resolves to a local file system target or a directory, it's not accessible in S3 Gateway.

• Relative symbolic links can be used normally under S3 Gateway.

• If you need to access an absolute symbolic link inside JuiceFS S3 Gateway, add --mountpoint to the start command, and specify the mount point.

  Assuming the mount point being /jfs, and the following symlink has been created inside JuiceFS:

  $ ls -alh /jfs
  file.txt -> /jfs/dir/file.txt

  Use the following command to ensure proper symlink resolution:

  juicefs gateway myjfs 127.0.0.1:8888 --mountpoint=/jfs