In big data scenarios, permission management for file systems and application components is critical. The newly released JuiceFS Community Edition 1.3 Beta 2 introduces integration with Apache Ranger, delivering a more flexible and fine-grained access control solution.
In this article, we’ll share how JuiceFS Community Edition integrates with Apache Ranger and explore its permission management implementation. This feature was contributed by Ming Li from the DMALL Data Platform Team, and we extend our gratitude for her contribution.
Overview of JuiceFS file system permission management
As a POSIX-compliant file system, JuiceFS manages permissions similarly to local file systems. Each file or directory is managed by users and groups:
- User and group management:
- Files or directories are assigned an owner user and owner group, stored in JuiceFS metadata as UID (User ID) and GID (Group ID) rather than usernames or group names.
- Since the same UID may map to different usernames across nodes, consistent UID and GID mappings must be maintained across all nodes.
- Access control lists (ACLs): Beyond basic user and group permissions, JuiceFS supports ACLs. ACLs enable granular permission control at the file or directory level, granting specific read/write permissions to individual users without requiring group membership.
JuiceFS integration with Apache Ranger
To simplify fine-grained permission management and enable centralized web-based administration, JuiceFS now supports Apache Ranger, a widely adopted security framework in the Hadoop ecosystem.
Apache Ranger overview
Apache Ranger is a centralized security management framework for Hadoop, providing fine-grained access control for HDFS, Hive, and HBase. Ranger facilitates policy management through its web interface, with all policies stored in a database. Components like the HDFS NameNode leverage Ranger's plugins to periodically fetch the policies and perform in-memory permission validation.
Integration architecture
The integration between JuiceFS Community Edition and Apache Ranger relies on Ranger's plugin mechanism. Since JuiceFS Community Edition lacks a server, all permission validation logic is executed on the client. The architecture is as follows:
- The Ranger plugin: Integrated into the JuiceFS client, it fetches permission policies from the Ranger server and performs local permission validation.
- Policy synchronization: To prevent excessive load on the Ranger Admin caused by multiple JuiceFS clients simultaneously fetching policies, JuiceFS implements an optimized policy. Only one client periodically fetches the policy from the Ranger server and writes it to a directory in JuiceFS. Other clients read the policy directly from the JuiceFS file system, reducing the pressure on the Ranger server.
- Permission validation workflow: When a user accesses a directory or file, the JuiceFS client checks whether there is a clear permission setting for the directory or file in the Ranger policy:
- If there is a clear allow or deny rule, it is executed according to the rule.
- If there is no clear rule, it falls back to the POSIX permission management mechanism (that is, permission verification based on UID or GID).
Configuration steps
The setup can be completed through simple JuiceFS commands:
# Specifies Ranger configuration during file system formatting.
juicefs format META-URL NAME --ranger-rest-url http://localhost:6080 --ranger-service jfs
# Adds Ranger integration to the existing file system.
juicefs config META-URL --ranger-rest-url http://localhost:6080 --ranger-service jfs
- Ranger server address: Specifies the Ranger server address. JuiceFS clients communicate with the Ranger server through this address.
- Ranger service name: Specifies the service name in Ranger. JuiceFS clients use this name to fetch the corresponding permission policy.
Notes
- Log management: The permission verification logic of JuiceFS Community Edition is completed on the client. The number of clients may be large. Audit logs are not currently supported.
- Security: Since the permission verification logic is executed on the client with open-source code, the permission enforcement mechanism is less stringent. Users can bypass permission verification by modifying the code or other means. Therefore, in actual use, management measures are required to ensure the effectiveness of permission control.
- Only Java client support: Currently, Ranger's client code is based on Java, so the integration of JuiceFS and Ranger is mainly applicable to Hadoop SDK.
Summary
The integration between JuiceFS Community Edition and Apache Ranger delivers a flexible, fine-grained permission management solution for big data file systems. Through Ranger's centralized policy management, users can efficiently configure access control rules for files and directories in JuiceFS. The integration between JuiceFS and Ranger delivers robust security safeguards for big data platforms.
